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This is the third book by 
Moore Special Tool Co. 

Moore Special Tool Company, Inc., 
of Bridgeport, Conn., U.S.A., has built 
a world-wide reputation since 1924, 
both as a manufacturer of special 
tooling to extremely close accuracies 
and of machine tools which make 
possible a very high degree of precision. 
In the latter category, its jig borers, 
jig grinders, measuring machines, 
and the accessories related to this 
equipment are considered to be among 
the best of its kind produced in the 
industrial world. 

A simultaneous characteristic of the 
company is its desire—and rare 
ability—to communicate the intimate 
knowledge it has acquired of these 
specialties to a wide audience. “The 
Moore Literature Library” is 
well-known within the industries it 
serves. Besides a wide array of 
brochures and case-history material, 
Moore in 1946 published a 448-page 
book. Predawn Hole Location, by 
J. Robert Moore, and, in 1954, Holes . 
Contours and Surfaces, by F. C. 

Victory and Richard F. Moore. These 
two books, devoted to jig boring and 
jig grinding, have sold over 40,101 
copies. The latter book is still in print. 

In this third book, Wayne R. Moore 
has assembled in the 350 pages of 
Foundations of Mechanical Accuracy 
the company’s Ultimate knowledge of 
and experience with mechanical 
accuracy, and how to achieve it. He 
has illustrated his text with over 501 
original photographs and drawings. 

This book tells how to attain precision 
in manufacturing to millionths of an 
inch and how to control such precision 
by appropriate measuring techniques. 

The book is divided into four main 
sections: geometry, standards of 
length, dividing the circle, and 
roundness. A fifth section covers 
“Universal Measuring Machine 
Techniques and Applications.” 


Introductory by M.I.T.’s 
Dean-Emeritus of Science 

The introduction is by Dr. George 
R. Harrison, Dean-Emeritus, School 
of Science, Massachusetts Institute of 
Technology, a world-renowned 
spectroseopist whose accurate 
diffraction gratings are used for 
measuring the lengths of light waves 
in the laboratory and in astronomy. 
Here are excerpts: 

“Basic to man’s behavior is his 
ability to determine, modify, and 
adapt to his environment. This he 
has been able to do in proportion to 
his skill at making measurements, 
and fundamental to all other 
measuring operations is his ability 
to determine locations in the material 
world. Thus the science of mechanical 
measurements is a fundamental one. 
It is this science, and the art which 
accompanies and informs it, with 
which this book is concerned.” 

After recounting his experience in 
working with the Moore Special TwI 
Company since 1959 to develop more 
accurate ruling engines for producing 
diffraction gratings, Dr. Harrison 
writes: 

"It is easy to see why the scientist 
who is primarily occupied with 
obtaining 100,000 straight grooves 
in a single operation, each a 
narrow mirror properly inclined to 
the beam of light which is to be 
reflected from them, welcomes most 
gratefully the development of fine 
mechanical machines which can, in 
an almost literal sense, be ‘bought 
off the shelves’. The amount of 
painstaking labor, skill in design, 
and the heritage of fine workmanship 
required before machines can be 
brought to such a stage of perfection, 
will be appreciated most of all by 
readers of this excellent book." 




Wayne R. Moore literally grew 
up with a passion for precision, which 
he inherited from his father, 

Richard F. Moore. In 1924, Dick 
Moore, a tool maker, had opened a 
two-man shop in Bridgeport to make 
special tooling for watch, clock and 
typewriter plants. It has grown to 
worldwide eminence 

Wayne joined the company in 1953 
after earning a degree in economics 
at Yale, and then completed his 
apprenticeship as a tool and diemaker. 
While at Yale, he captained, in 
1952-53, what has been called the 
greatest swimming team of Bob 
Kiphuth, famous Eli coach. He won 
gold medals in both intercollegiate 
championship races and in the 1952 
Olympics at Helsinki. 

The research and writing of this 
book, as well as the time he devoted 
to supervising the extensive 
illustrations, were sandwiched in over 
a period of 7 years while he has been 
active in helping the Moore Company 
develop its advanced metrology and 
machine tool products. 

He resides with his wife, Janice, 
and five children in Nichols, 
Connecticut, where he is a dedicated 
organic gardener. 
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Preface 


A technical book devoted to a little-ex¬ 
plored subject originates long before it is 
actually written. Many people contribute 
to its content through their guidance and 
assistance. This book is no exception. 

I consider myself fortunate to be asso¬ 
ciated with a company whose policy has 
always been to share both its processes and 
its techniques. To make available the re¬ 
sources of Moore Special Tool Company 
for the improvement of industry, science, 
and society in general has been the philos¬ 
ophy of my father, Richard F. Moore, who 
founded the company in 1924. The policy 
has been rewarding to the company in 
many ways. I am grateful that I have the 
opportunity of continuing in what has be¬ 
come a Moore tradition. 

In 1946, Moore published its first full- 
length book, Precision Hole Location for 
Interchangeability in Toolmaking and Pro¬ 
duction, by J. Robert Moore. In 1954, 
Holes, Contours and Surfaces, by Richard 
F. Moore and Frederick C. Victory, 
followed. 

This volume was planned to explain in 
graphic detail the significant course which 
mechanical technology has taken since 
Holes, Contours and Surfaces appeared. 
This course is directed inexorably towards 
greater accuracy. I undertook the task 
largely because of Richard F. Moore’s en¬ 
couragement, guidance and support. Most 
of the mechanical principles described 
herein can be traced to him. 


The significance of the World’s Bureaus 
in establishing and maintaining standards 
is described in a special section, Pages 95- 
97, written especially for this book by an 
outstanding metrologist of each Bureau: P. 
Carre, of the Bureau International des 
Poids et Mesures, France; L. W. Nickols, 
of the National Physical Laboratory, En¬ 
gland; E. Engelhard, of the Physikalisch 
Technische Bundensanstalt, West Ger¬ 
many; A. G. Strang, of the National Bu¬ 
reau of Standards, U.S.A. They made pos¬ 
sible a description of the functions of their 
sections of each Bureau, met with the au¬ 
thor personally for helpful discussions, and 
also contributed valuable additional ma¬ 
terial. Their colleagues—Stanley P. Poole, 
of NPL; Rudolf Noch, of PTB, and T. R. 
Young, of NBS—also provided useful in¬ 
formation. 

I warmly thank T. R. J. Oakley, of NPL, 
who not only reviewed the entire manu¬ 
script and illustrations meticulously, and 
gave innumerable suggestions and com¬ 
ments. It was my good fortune to enlist 
for this important role a man who soon 
confirmed that I could have selected no 
one with a greater depth of knowledge and 
experience in the subject. 

Richard L. Parnoff, the Chief Inspector 
at Moore, also reviewed the manuscript 
and the illustrations. He brought to the 
project the same thoroughness and valu¬ 
able suggestions that aided the authors of 
Holes, Contours and Surfaces. 

The Moore Company and I are singular¬ 
ly honored to have Dr. George R. Harri¬ 
son, Dean Emeritus of Science of the Mas¬ 
sachusetts Institute of Technology, write 


the Introductory. His encouraging words 
and advice, including the refining of the 
book’s title, were also in a personal sense 
very important to me. 

Robin Gosling, of the Science Museum, 
London, and Bernard Bothmer, Curator of 
Ancient Art, Brooklyn Museum, led me to 
many historical works. I was able to find 
items of historical interest with the help of 
Gilbert L. Dannehower, of High Precision 
Products Company; George Schaffer and 
Rupert LeGrand, of American Machinist, 
and Mrs. Nettie Wright Adams, editor and 
publisher, The Lure of Litchfield Hills. 

Many firms, institutions and individ¬ 
uals, whose names are credited throughout 
the pages, contributed to the content of 
the book. 

Years of exposure of the Moore staff to 
world-renowned metrologists, who came to 
Bridgeport in their pursuit of accuracy, 
greatly enhanced the scope of the book. 
I single out particularly James Bryan, of 
the Lawrence Radiation Laboratory, Uni¬ 
versity of California, with whom many in¬ 
formative discussions were held. 

Alfred Sturzenegger, of the Precima 
Company, West Germany; Robert Mun- 
dy, of the Catmur Corporation, England; 
Claude Pardessus, of Stokvis et Fils, 
France; Walter Zindel, of the Moore Office 
in Switzerland, and Arvid Arvidsson, of 
Sweden, all provided helpful liaisons within 
their countries. 
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So many of my colleagues at Moore 
Special Tool Company assisted in bringing 
this project to a successful conclusion that 
space doesn’t permit listing all by name. 
Albert Johnson, chief engineer of special 
products, and Anto Lindberg, chief engi¬ 
neer of standard products, and their staffs; 
Richard Kuba, advertising coordinator, 
gave much time and interest to it. Mrs. 
Justine Haydu not only typed and re¬ 
typed the manuscript with unfailing good 
humor, but managed to organize a moun¬ 
tain of source material, pursue the bibliog¬ 
raphy and maintain the paperwork so effi¬ 
ciently as to permit a book of this size and 
scope to be researched in an office con¬ 
cerned with the engineering and sale of 
machine tools. 

My deep thanks go to numerous Moore 
employees, some of whom appear in illus¬ 
trations throughout the book and others 
behind the scenes, for their cheerful co¬ 
operation in so many ways. 

The writing itself took place in the quiet 
confines of the company’s legal offices, the 
chambers of Charles Brody and Seth 
Brody, old friends and advisors. To the 
Brodys, their partners and staff, my grate¬ 
ful thanks for the splendid retreat they 
provided. 


Since the Moore Company itself is pub¬ 
lishing this book, we also required all the 
disciplines associated with publishing. For 
this, I was fortunate to have the services 
of the company’s long-time marketing and 
communications counsellors, The Fred 
Wittner Company, which had guided both 
previous books to successful completion. 
Fred Wittner helped to shape the direction 
of the book and to edit it for publication; 
he also contributed sound advice, based on 
his great breadth of experience. Sel Torby 
designed it and supervised its production 
with understanding, good taste, and con¬ 
summate skill and patience. 

All of the color and black and white 
photography taken at Moore is the work 
of William Vandivert, a resourceful journal¬ 
ist-photographer, who was a Life Maga¬ 
zine “great” for many years, and who also 
proved to be a discerning critic and coun¬ 
sellor. The engineering drawings are the 
product of F. R. Gruger, Jr., outstanding 
technical illustrator, who did the same 
work for Precision Hole Location and Holes, 
Contours and Surfaces. 

Wayne R. Moore 

Bridgeport, Connecticut, U.S.A. 

July 22, 1970 
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Introductory 

Basic to man’s behavior is his ability to 
determine, modify, and adapt to his 
environment. This he has been able to do 
increasingly as his skill at making measure¬ 
ments improved, and fundamental to all 
measuring operation is his ability to deter¬ 
mine locations in the material world. Thus 
the science of mechanical measurements is 
a basic one. It is this science, and the art 
which accompanies and informs it, with 
which this book is concerned. 

I am particularly happy to be invited to 
write these introductory words because it 
gives me an opportunity to testify to the 
tremendous importance of the availability 
of machines which make precise and ac¬ 
curate mechanical measurements possible. 
My acquaintance with the Moore Special 
Tool Company ended a quest for instru¬ 
ments of the utmost mechanical refinement, 
which could then be refined still further 
by the application of optical and electronic 
controls in ways not previously attempted, 
because the technologies needed for such 
attempts had not previously been available. 

As a spectroscopist interested in measur¬ 
ing to seven significant figures the lengths 
of light waves, themselves only 10 to 20 
millionths of an inch long, I was seeking 
ways of ruling bigger and better diffrac¬ 
tion gratings. With these the light waves 
from atoms and molecules, whether in the 
laboratory or in the stars, could be more 
adequately studied. 

The problem is simple to outline. On a 
glass or fused quartz mirror blank coated 
with a thin layer of aluminum deposited in 
vacuum, one desires to engrave 100,000 
long grooves, each straight to a millionth 
of an inch, parallel to its neighbors to with¬ 
in that tolerance, and equally spaced, so 
that every groove is within a millionth of 
an inch of its proper location. Furthermore, 


any repeated or periodic errors in groove 
location should be held to less than one- 
tenth of a millionth of an inch. 

Up until 1947 these specifications had 
been fairly well met for gratings up to 6 
inches in width by purely mechanical ruling 
engines, builtin scientificlaboratories, usual¬ 
ly in a cooperative effort between a physical 
scientist and a superbly skilled instrument 
maker. But many experts had spent long 
years in futile attempts at ruling wider 
gratings with longer grooves. Some had felt 
that the way to solve this problem was by 
making light waves themselves correct the 
errors of the engraving machine during the 
ruling process, but the proper technologies 
for carrying this out were not yet available. 

After having taken a solemn oath to my¬ 
self never to get entangled with the unend¬ 
ing problems of grating ruling, I decided in 
1947 that the newly available mercury 
isotope lamps might furnish light of a suffi¬ 
cient singularity of color to guide such an 
engine. When the University of Chicago 
offered M.I.T. an old, warped, partially 
rebuilt engine whose base castings had 
been machined under the direction of A. A. 
Michelson back in 1900, I felt that if inter¬ 
ferometric control could make this device 
work, it could make any engine work. 

The next ten years were spent in devel¬ 
oping optical control methods and servo¬ 
mechanisms which did, indeed, reduce the 
errors of positioning of the engine ruling 
tool to l/600th of their previous magni¬ 
tude. With this so-called “A” engine, de¬ 
scribed in the last section of this book, my 
colleagues and I were able to rule many 
gratings of 10-inch width and of a quality 
never previously attained. 

In 1959 we were ready to build a second, 
larger engine, and I began a search for 
producers of very accurate screws. It 


seemed of great importance to demon¬ 
strate that a commercially built machine, 
placed under interferometric control, could 
be made to function as a ruling engine. The 
Moore Special Tool Company had avail¬ 
able drive screws which were nine times 
more accurate than that used in our A 
engine, and after testing two or three of 
these we ordered a Moore Universal 
Measuring Machine of 18-inch capacity, 
without the superstructure. When placed 
under interferometric control this proved 
much more accurate than the “A” engine, 
and with it many outstanding gratings up 
to 16 inches in width, with grooves up to 8 
inches long, have been ruled. 

As recounted later, when this engine was 
successful we ordered a No. 4 measuring 
machine from Moore, have now put this 
under interferometric control, and have 
ruled a number of gratings in sizes up to 
12 x 15 inches. We hope eventually to pro¬ 
duce 18 x 24 inch gratings with it. 

From this story it is easy to see why the 
scientist who is primarily occupied with 
obtaining 100,000 straight grooves in a 
single operation, each a narrow mirror 
properly inclined to a beam of light which is 
to be reflected from them, welcomes most 
gratefully the development of fine mech¬ 
anical machines which can, in an almost 
literal sense, be “bought off the shelves.” 

The amount of painstaking labor and 
skill in design, and the heritage of fine 
workmanship required before machines 
can be brought to such a stage of perfec¬ 
tion, will be appreciated most of all by 
readers of this excellent book. That it will 
be welcomed by a wide range of readers 
goes without saying. 

George R. Harrison 

Dean Emeritus, School of Science 

Massachusetts Institute of Technology 
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“. the architect in his work ought to be 

practiced in all accomplishments. Yet reason, 
in view of the scope of matters, does not permit us, 
as need demands, to have a complete, but only a 
moderate knowledge of the various subjects involved. 
Hence I beg your Highness and other readers of 
these volumes to pardon any explanation that too 
little agrees with the rules of literary art. For it is not 
as a lofty thinker, nor as an eloquent speaker, 
nor as a scholar practiced in the best methods of 
literary criticism, but as an architect who has a mere 
tinge of these things, that I have striven to write 
the present treatise. But in respect to the meaning 
of my craft and the principles which it involves, 

I hope and undertake to expound them with assured 
authority, not only to persons engaged in building, 
but also to the learned world.” 

—Vitruvius, “De Architectura” 
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fig. 1 — The attainment of a “cubic 
concept" of accuracy in the Universal 
Measuring Machine requires the 
individual mastery of four mechanical 
arts — geometry, length, dividing the circle 
and roundness. 


Throughout the text, an attempt will be 
made, where relevant, to immediately 
accompany each given dimension with the 
equivalent figure in metric or English units, 
whichever the case may be. 

The converted figure will not always 
be exact, but will be rounded off to what 
seems to be the nearest significant figure. 

This will be done with a special 
bracket used only for this purpose. 

For example: 

10 millionths of an inch [0.00025 mm] 
100 mm [3.937 inches] 

20°C. [68°F.[ 

Itishopedthat this will provide 
greater ease of reading to those who 
might be more familiar with one or the 
other system. 

However, in those cases where the 
converted figure detracts from clarity, 
or does not aid in understanding, it will, 
be omitted. 
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We are confronted today with a bewilder¬ 
ing array of measurement hardware and 
measurement techniques. However neces¬ 
sary, they tend to obscure the beautiful 
and fundamentally simple nature of the 
measurement process itself. 

As did Vitruvius, we recognize our in¬ 
complete knowledge of the total subject 
involved, and certainly a lack of eloquence. 
Yet, as builders of tools, we find ourselves 
uniquely qualified to “undertake to ex¬ 
pound with assured authority” the meaning 
and significance of measurement . . . both 
to show it reduced to its simplest elements 
and to explain how these, in turn, relate not 
only to advanced mechanisms, but to the 
very frontiers of measurement. 

The frontiers of measurement are criti¬ 
cally related to the progress of civilization 
itself, notably in interchangeability of 
manufacture. When brought to their high¬ 
est level, the concepts described here con¬ 
tribute to extending the frontiers of scien¬ 
tific knowledge to the outermost limits of 
the universe. As an example, these princi¬ 
ples have been applied to the manufacture 
of ruling engines used to produce the most 
accurate gratings now available—gratings 
that are mated with giant astronomical 
telescopes to analyze light from the most 
distant stars. 

The concept of measurement which un¬ 
folds derives largely from our own experi¬ 
ence. Because of this approach we may 
dwell too lightly or even neglect certain 
aspects which others may think more sig¬ 
nificant; yet it is hoped that the gain to the 
reader will be greater continuity and a 
fuller comprehension of the total concept 
of measurement. 

At Moore, a goal had been set to build a 
machine tool to a cubic concept of accu¬ 
racy.* It would measure by coordinates 
to a much closer tolerance than had previ- 

*Richard F. Moore & Frederick C. Victory, Holes, 
Contours and Surfaces, Moore Special Tool Company, 
1955, p. 56. 


ously been achieved. The cube would have a 
tolerance of approximately 25 millionths of 
an inch in an 11 by 18 by 17 inch travel 
[0.0006 mm in 279 by 457 by 432 mm]. 

The reader may appreciate the difficulty 
of this task when reminded that the toler¬ 
ance of a single gage block of the highest 
laboratory quality is +0.000001 (1 mil¬ 
lionth of an inch) per inch of length 
[0.000025 mm per 25 mm] or approximately 
18 millionths of an inch over 18 inches 
[0.00046 mm over 457 mm] (if indeed a gage 
block could be built to such a tolerance in 
larger sizes). 

Besides, this was not to be just a “static” 
cube, but rather a “live” cube, within 
which the ordinates must provide this po¬ 
sitional accuracy of 25 millionths of an 
inch, [0.0006 mm] including straightness 
and squareness over an infinite number of 
intervals within the cube, Fig. 1. 

Existing in-shop standards and masters 
were not capable of attaining the desired 
accuracies. A complete re-evaluation led to 
one conclusion: We had to go back to our 
fundamental standards and build from 
there. 

This process forms the basis of our story. 

To achieve this new generation of ac¬ 
curacy and to incorporate it to the maxi¬ 
mum in a Universal Measuring Machine 
required the individual mastery of four 
mechanical “arts.” 

1. Geometry 

2. Standards of Length 

3. Dividing the Circle 

4. Roundness 

1. Geometry: The total geometry of the 
machine tool has its foundation in the 
flat plane. The familiar surface plate is 
its embodiment. From the surface plate 
evolves straightedges, squares, scraping 
masters and machine ways. 

2. Standards of Length: Here the em¬ 
phasis is on where the measuring ele¬ 


ment of a machine tool—in Moore’s 
case the lead screw—derives its accu¬ 
racy. In the process of arriving at the 
lead screw system, all possible forms of 
length-measuring standards come under 
consideration, such as: gage blocks, end 
standards, precision scales and light¬ 
wave measuring interferometers. 

3. Dividing the Circle: Accurate circle 
division is a requirement of all labora¬ 
tories and machine shops. It is also 
closely related to the cubic concept of 
accuracy since a machine tool of the jig 
borer family must measure by coordi¬ 
nates as well as have the facility to 
make angular measurements. 

4. Roundness: Precision machine tools 
must achieve roundness of many me¬ 
chanical parts and especially with their 
spindles. The focal point of roundness 
here is the Universal Measuring Ma¬ 
chine spindle. Its performance, in turn, 
is dependent on the overall accuracy of 
holes, shafts, balls and other compo¬ 
nents which comprise its construction. 

These four mechanical arts are truly the 
“Foundations of Mechanical Accuracy,” 
since all forms of measurement can be re¬ 
solved into only these four fundamentals. 

Geometry, dividing the circle and round¬ 
ness need no reference to an outside au¬ 
thority; the degree of accuracy depends 
only on the technique, facilities and skill of 
the maker. Standards of length, on the 
other hand, are arbitrary, so are ultimately 
derived from some authoritative source, 
such as the N ational Bureau of Standards 
of the United States, the National Physical 
Laboratory of England, the German Stan¬ 
dards Bureau, or the International Bureau. 
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GEOMETRY 



The foundation of the mechanical art of 
geometry is the flat plane. Evolving 
from the flat plane are master straight¬ 
edges and squares, scraping masters, 
laps, and finally the geometric accuracy 
of the machine-members themselves. 


PIG. 5 —The foundation of all geometric 
accuracy and indeed of all dimensional 
measurement is the flat plane. 

Flatness is inherently attained by the 
cross matching of three surface plates. These 
master 4 . 8 -inch [1219 mm] surface plates 
are made of stabilized iron, are square and 
extremely rigid because of their box-type 
construction. 











ENVIRONMENTAL EFFECTS 
ON CAST IRON 


Preliminary to discussing the mechanical art of geometry, some common 
misconceptions about cast iron, a prime material component of most of the 
elements of geometry, must be dispelled. 

Also analyzed here are the principles of recommended inspection procedure. 
In the discussion of geometry that follows, frequent references will be made 
to these principles. 


It is not uncommon for a cast iron part, 
whether straightedge, surface plate, or 
machine tool, to seem to lose its straight¬ 
ness, flatness, squareness, or to change in 
some other manner over a period of time. 
The stability of the iron is often blamed 
when, in fact, this is rarely the case. 

This section has a threefold purpose: 

1. To show that cast iron is inherently 
stable; 

2. To differentiate between real and 
apparent sources of instability; 

3. To point out those environmental 
conditions essential to geometrical 
accuracy. 



FIG. 2 —The conclusion that cast iron is 
inherently stable is based partly on strong 
statistical evidence—that of thousands of 
castings machined and scraped to a 
higher order of accuracy as well as careful 
records kept over many decades. 
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1 . Foundry Technique 

Foundry technique is a factor in stability 
and hence in final accuracy. 

A. Proper composition of the iron im¬ 
proves machinability, wear qualities, 
and the quality of the scraped sur¬ 
face (it is physically difficult to ac¬ 
curately scrape a bad casting or one 
with blow-holes). 

B. Stability of the iron depends mostly on 
slow, uniform cooling in the mold after 
casting. This, in turn, depends on 
molds of the proper design, and on 
allowing sufficient time for complete 
cooling before the casting is exposed 
to air. There is possibly no other met¬ 
al more stable than cast iron that 
meets the preceding two specifica¬ 
tions. 

Our faith in the stability of iron derives 
from strong statistical evidence: the expe¬ 
rience of thirty years of precision machine 
tool manufacture, involving, at the time of 
writing, over 7,000 machines, Fig. 2. It 
also derives from the study of a jig grinder 
which had been returned for repairs after 
having been in a severe fire in an automo¬ 
tive plant. Completely scorched, its paint 
had been burned off and it was caked in tar 
and soot. The temperature was high enough 
to melt the aluminum guards into a puddle 
on the table. 

The component cast iron parts were first 
immersed in a pit of solvent for cleaning, 
then inspected. There was virtually no de¬ 
viation from the original factory calibra¬ 
tion. Any lingering doubts we might have 
had about the stability of cast iron were 
dispelled. 

The inherent stability of cast iron was 
also confirmed by Nickols of the N.P.L., 
who made tests on its stability over a pe¬ 
riod of seven years.* 

*L.W. Nickols, “Investigations into the Stability 
of Castings (with particular reference to Meehanite 
castings),” NPL Technical Bulletin S.S. 193, May, 
1940. 


For the bases, columns, tables and cross 
slides of the machine tools it builds, as well 
as for surface plates, straightedges and 
scraping masters, Moore uses only a Mee¬ 
hanite cast iron, close-grained, of closely 
controlled analysis and cooled slowly in the 
mold. 

II. Machining of Cast Iron 

Despite its stability, cast iron does not 
differ from many other metals in that the 
removal of a substantial amount of ma¬ 
terial by machining, especially the outer 
skin, will alter its equilibrium. Before the 
finish cut, the piece is first allowed to relax 
by release of the clamps; it is then properly 
supported and re-clamped without bend¬ 
ing, Fig. 3. 

The piece may or may not be free of 
stresses, depending on the method by which 
it was machined. 

The preferred method of machining is 
to single-point plane important functional 
cast iron surfaces, Fig. 4. The reasoning 


behind this procedure may be seen in a con¬ 
trolled experiment conducted to determine 
the effects of machining on cast iron. Using 
cast iron gibs as test pieces, it was found 
that with single-point planing, the gibs 
would spring the least from machining and 
would be absolutely stable henceforth. 
Milling the gibs caused more machining 
distortion and a somewhat permanent in¬ 
stability until the gibs were stress-relieved. 
In other words, it is possible to machine-in 
instability. 

III. Elastic Deflection 

A large cast iron section will easily bend 
when a load is applied, or when weight is 
added, or may even bend under its own 
weight. So often and in so many ways does 
elastic deflection occur, that it is easily mis¬ 
taken for instability of the iron itself. De¬ 
flection may be minimized through design 
and support, but never completely elimi¬ 
nated. In some instances, as the reader will 
observe with the two-footed gage (page 26), 



fig. 3 —For maximum accuracy when 
machining surfaces, irregularities in the 
supporting surface are balanced out 
through the use of shims. Most of the 
heat and stress of machining is removed 
in a first rough-cut. 


The workpiece is unclamped, permitting 
it to relax. It is supported evenly once 
again, clamped less tightly and finished 
with a light skim cut. 
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deflection is not necessarily detrimental to 
accuracy, since the amount of deflection is 
almost perfectly constant, given like condi¬ 
tions. In the case of the extremely rigid 
48 inch [1219 mm] master surface plate, a 
small amount of deflection still occurs, and 
will be a temporary limiting factor in the 
attainable accuracy of the plate. 

It is important to recognize that, with 
every step in the attainment of accuracy, 
the problem of elastic deflection is always 
present. This fact will continually be em¬ 
phasized throughout the book. 

IV. Temperature and Instability 

A. Geometric relationships are not effected 
by the value of ambient temperature, 
providing it is steady. For example, a 
surface plate, flat at a temperature of 
70°F. [21.1°C.], will be flat at a tem¬ 
perature of 90°F. [32.2°C.[. A test 
square, in error by .2 seconds of arc at 
90°F., will be in error by .2 seconds 
of arc at 70°F. [21.1°C.[. In a ma¬ 
chine tool, two female V’s of one 
member that match two male V’s of 
another member where both are the 
same temperature, will also match 
each other when both are some other 
temperature. The distance between 
the V’s in each will change, but by 
the same amount. 

B. Temperature Differentials —If, how¬ 
ever, the piece or pieces are subject 
to localized heat or cold, not only 
does growth or shrinkage occur, but 
the piece may be subject to unpre¬ 
dictable distortion. It returns to its 
normal geometric relationship only 
when the temperature throughout is 
again made uniform. The distortion 
which occurs as a result of uneven 
temperature is the source of a great 
deal of misunderstanding and confu¬ 
sion. The result so resembles that 
which one would expect from “un¬ 
stable” iron that it is not surprising 













the iron is blamed. Imagine a situa¬ 
tion in which a piece had been 
scraped, inspected and, unknowing¬ 
ly, proved accurate in a condition of 
non-uniform temperature. When the 
piece returns to uniform temperature, 
attempts to repeat the previous accu¬ 
racy check are in vain. 

Some persistent, recurring sources 
of temperature differentials are worth 
mentioning: 

—Proximity to radiators, doors, win¬ 
dows or drafts; 

— Heat from handling; 

— Changes in room temperature. 
This may particularly effect larger 
pieces, because the time required 
to “soak out” may be several hours. 
—Stratification—It is extremely dif¬ 
ficult to control the temperature of 
a room accurately from floor to 
ceiling. Even in advanced metrol¬ 
ogy laboratories, where tempera¬ 
ture can be controlled to ± J4°F. 
[% 4 °C. approx.] within a height of 
two or three feet [0.6 to 0.9 meter] 
there will usually be a difference of 
2°F. [1.1°C.] from floor to ceiling. 
This is satisfactory for small work, 
but the average machine tool stands 
at least six feet [1.8 meters] so will 
be subjected to temperature gradi¬ 
ents of the room. (This effect on 
linear measurements will be dealt 
with on pages 118, 119 and 171, 
172). Temperature control is, of 
course, the best answer but will be 
of little value unless certain correct 
procedures are also followed. 

Conclusion 

The theory that a cast iron part must be 
exposed to the weather to rust and “age” in 
order to stabilize it is a carry-over from the 
past. In the absence of closer measuring 
facilities or adequate temperature control, 
the stability of the iron was blamed. Sup¬ 


posed cases of instability might have been 
traced to machining practices, deflection 
or, most often, to temperature variations. 

Cast iron is an ideal material. It is readily 
available, can be easily cast to shape, and 
is relatively uninfluenced by humidity. It 
has nearly the same coefficient of expan¬ 
sion as steel and will give a bearing to 
which a piece may be scraped. Above all, it 
is one of the most stable of materials. These 
advantages are not present in all steels, 
granite, or other materials which might be 
considered as a substitute. 

Requirements of an Inspection Tool 

The requirements of an ideal inspection 
tool are five-fold. They are analyzed here 
in advance of the discussion of geometry 
because they all pertain to the inspection 
principles Moore espouses: 

It is accurate; 

It requires a minimum of operator skill; 

It inspects a specific type of error; 

It is fast to use; 

It is self-checking. 

The first two requirements are readily 
understood: the degree of accuracy of cali¬ 
bration depends on the accuracy of the in¬ 
specting instruments; devices which reduce 
dependence on operator skill contribute 
to both efficiency and accuracy. 

Inspection should not just show that 
“something’s wrong.” This only leads to 
costly checking, misunderstandings, and 
greater chance for error. It is sounder to 
inspect for specific error; e.g., deviation 
from a horizontal straight line travel of a 
machine’s ways. Nothing should be as¬ 
sumed to be correct and left uninspected. 
For example, if the deviation from a hori¬ 
zontal straight line travel of a machine’s 
ways has been inspected, it does not follow 
that the vertical straight line travel has 
also been inspected. 

The requirement of speed in inspection is 
not generally appreciated. Inspection must 


be performed fast for more than just eco¬ 
nomical reasons. Where inspections become 
“involved,” where there is a time lapse be¬ 
tween start and finish of a check, the more 
assuredly will errors creep in from temper¬ 
ature change—change in the datum of the 
calibration instrument, “re-picking up,” or 
from the necessity of lengthy calculations. 
Even under the best of temperature-con¬ 
trolled conditions, body heat alone can 
still make metal bend, twist, or grow. This 
can cause confusion if not understood. 

Finally, a good inspection tool should be 
capable of being checked against itself. 
Ideally, in exploring for a specific type of 
error, a method should be used which each 
time simultaneously re-checks the gage. If 
this is possible, the inspection method 
gains much in reliability. 
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fig. 6 —The classical method of creatmg a 
flat plane ■is based on the cross matching of 
three plates, such as “A,” “B” and “C” 
[top). The techniqxie is unrelated to the 
initial accuracy of the plates. “A” is 
scraped to match “B” (middle left) and also 


to match “C” (middle right). At this point, 
only conformity has been achieved. When 
“B” is compared to “C” (Joiver), error is 
revealed. 





fig. 7— By systematically matching three 
surface plates and scraping the high points 
of their contact, three flat planes are created, 
regardless of initial flatness of the plates. 
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FIG. 8 —Fig. 6 is only partially trite. 

The classical three-plate method does not 
apply when the plates are rectangular. 
Two rectangular plates are shown with an 
identical exaggerated “twist” (opposite 
diagonals are high and low). Although the 


plates are not flat, they will match perfectly, 
even if turned end-for-end. This is 
obviously also true if a third plate of 
identical shape is introduced. 


1 . THE FLAT PLANE 

The classical method of creating a flat 
plane is based on the use of three plates. 
The procedure is simple: 

When two plates are not flat but still 
match, one will not match the third. By 
continually lapping or scraping the high 
points of their contact until all three show 
perfect bearing when intercompared, three 
flat planes are created, Fig. 6 and Fig. 7. 

Rectangular plates may have a “twist,” 
Fig. 8, and still match each other to show a 
perfect bearing even if turned end-for-end, 
yet none of the plates may be flat. A square 
plate turned 90° on its mate would reveal a 
twist, Fig. 9, but it is not practical to do 
this with a rectangular plate since the over¬ 
hang would introduce as much error as it 
was intended to discover. 

The master flat planes illustrated in 
Fig. 5, consist of 48 inch [1219 mm] sur¬ 
face plates of stabilized cast iron. They are 
of box-like, or sandwich, construction with 
strong internal ribbing, Fig. 10. This makes 
them many times more rigid than conven¬ 
tionally-designed surface plates, Fig. 11. 

Awareness of the potential rigidity in 
this design came from a technical report* 
on the rigidity of a welded steel surface 
plate, fabricated of two steel plates with 
internal steel ribbing of an overall thickness 
to resist bending. This design is much more 
rigid than conventional surface plate con¬ 
struction. The reader may prove this to his 
own satisfaction by a practical experiment. 
Simply take a flat piece of wood and add 
only ribbing and four sides to it. Try bend¬ 
ing this “conventional” design.Next, attach 
a second flat piece to form a sandwich, and 
try to bend the assembly. Rigidity is in¬ 
creased many-fold. 

However much rigidity is desirable, the 
possible instability of a weldment rules 
against having this design as a master. Not 

*Adolph Kleinsorge, “Welded Surface Plates,” 
Bulletin 60-WA-241, American Society of Mechani¬ 
cal Engineers, New York. 



FIG. 9— A square plate will reveal a twist, 
since it may be turned through 90° and 
applied once again. 
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FIG. io— 2600 lbs. [1180 kg.], box-type 
surface plate, with internal ribbing, is 
many times more rigid than conventional 
surface plates. Flatness that can be 
achieved by the three-plate method is for the 
most part, dependent on rigidity. 




FIG. 11— I,2-inch [1067 mm] square 
surface plates, designed with deep external 
ribbing, and supported on three points, one 
of which is a balance beam—were first 
adopted as the master flat plane. 

The superiority of the box construction 
later became apparent. 


fig. 12 —The degree of flatness which can 
be attained with the master 1,8-inch 
[1219 mm] surface plate, and also its 
accuracy in use, depends on the type of 
support it receives. 

Three-point support is used for geometric 
stability. The single-leg side uses a balance 


beam, taking the weight from the corners 
of that side. To further distribute the weight 
and to minimize bending, each of the four 
support points in the corners is itself a 
three-pad contact. This provides a total of 
12 support points, each placed under the 
corner of a rib. 









FIG. 13 —To prevent stress, each of the four 
support points of the master, box-type 
surface -plate is free to pivot. Arrangement 
of the 12 contact points, each under the 
corner of a rib, is shown. 


Side lifting plate is attached through use of 
shoulder bolts, which provide clearance to 
the. side plate all around, minimizing both 
boiling and lifting stresses. 


until a similar construction of cast iron was 
seen at the Dixi company (Le Locle, Switz¬ 
erland), was it realized that such a con¬ 
struction was feasible as a solid casting. 
Complexity of the pattern and difficulty 
in casting both proved to be much less 
hindrance than first anticipated. Accord¬ 
ingly, six 48-inch [1219 mm] surface plates 
were designed and cast. All the rigidity that 
could be desired, plus the stability inherent 
in cast iron, was found in this design. 

The basic design underwent further evo¬ 
lution. The plate was supported on its 
underside at three points for geometric 
stability. A “two-legged equalizer” on the 
single-point support takes the weight from 
the corners of one side. To further distribute 
the weight and to minimize bending, each 
of the four support points in the corners 
serves as a three-pad contact. Each pad 
is, in turn, located directly under a corner 
of a symmetrically-placed ribbing arranged 
in squares, Fig. 12. 

While this construction has the advan¬ 
tage of the geometric stability of three- 
point support, note also, Fig. 13, that the 
weight of the surface plate is distributed on 
twelve points to minimize local bending— 
especially important when carrying the 
weight of large workpieces, machine mem¬ 
bers, or scraping masters. Resistance to 
bending is also necessary in achieving its 
initial accuracy, since it must support its 
mate in the process of cross-matching and 
scraping. 

The 48-inch [1219 mm] surface plates 
have l /i inch [12.7 mm] thick internal rib¬ 
bing and inch [19 mm] side thickness. 
Top and bottom surfaces are of one-inch 
[25.4 mm] thickness. Each surface plate 
weighs 2600 pounds [1180 kg.]. 




FIG. 14— The round shape of the granite 
surface plate is also geometrically correct 
for establishing flatness. Granite surface 
plates are the most familiar form of the flat 
plane and are convenient for many classes of 
inspection. The surface plate shown is flat 
to 50 millionths of an inch [0.0013 mm}. 
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FIG. 15— Surface plates “A,” “B” and 
“C” are tivisted, having opposite diagonals 
high, “H,” and low, “L.” When matched 
as shoivn (top), the bearing appears 
uniform. When any one of the plates is 
rotated and applied once again (bottom), the 
bearing pattern then appears as illustrated. 






Creating a Master Flat Plane 

The 48 inch [1219 mm] surface plates are 
square. Most surface plates in use today 
are rectangular, having something like a 
4:3 relationship (which is the usual longi¬ 
tudinal to cross-travel relationship in ma¬ 
chine travel). Square or round plates are 
geometrically most correct for creating a flat 
plane by the three-plate method, Fig. 14. 

To create a Master Flat Plane, the pro¬ 
cedure is to systematically cross-match 
three 48 inch [1219 mm] cast iron surface 
plates as well as to successively turn them 
90° one to another to reveal “twist,” Fig. 
15. Red rouge is commonly used on the 
plate which, though in error, is periodically 
chosen as the master, and a lighter orange 
rouge used on the plate to be scraped. The 
result is a pleasant, visible contrast of dark, 
high spots on a soft, non-glare background. 

Initial deviations from flatness are usu¬ 
ally considerable. The hills and valleys, 
Fig. 16, are kept low and wide while rough¬ 
ing by using a wide, flat scraper blade and 
long, deep cuts until some uniformity is 
achieved, Fig. 17. 

A liberal use of rouge is found to be de¬ 
sirable at first, but the difference between 
smears, or “false bearing,” is noted, and 
emphasis is placed on cutting the areas of 
dark, glossy spots. The rouge actually has 
size, so it is wiped thinner and drier as the 
bearing closes in to insure almost direct 
contact of plate to plate without any false 
bearing. As the bearing is closed in, it be¬ 
gins to have a “salt-and-pepper” appear¬ 
ance. As the spots become progressively 
more uniform and evenly spaced, a nar¬ 
rower blade is used, the angle of cut is 
raised gradually from the horizontal, and 
the stroke is shortened. 

The type of scraping stroke found most 
satisfactory is the “half moon,” Fig. 18. 
The advantage is that the stroke starts 
from the valley, reaches its greatest in¬ 
tensity directly on the spot (plateau), and 


fig. 16 —The “bearing” of a scraped 
surface is a series of uniformly high 
plateaus, which are ivide apart when 
scraping begins. After repeated scrapings, 
the bearing comes closer and closer together 
(top to bottom). 
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fig. 17— A skilled, patient scraping hand 
can achieve almost unbelievably accurate 
surfaces purely by scraping to the bearing 
imparted, and without the use of auxiliary 
inspection equipment. 


fig. is —Dark areas are plateaus of 
bearing, the lighter areas the valleys in 
between. The “half-moon” stroke starts from 
the valley, reaches its greatest intensity 
directly on the plateau and again fades to 
nothing as it passes to the other side. 












again fades to nothing as it passes to the 
other side. The type of cut used also de¬ 
pends on the purpose for which the surface 
is to be used. For instance, a surface to be 
used as a master for bearing should be cut 
deep for longer life, whereas the cut should 
be shallow on the ways of a machine where 
rollers are to be used for smoother rolling 
action. 

The scraping procedure takes consider¬ 
able skill and the experienced scraping 
hand is a precious resource. A novice can 
easily “break up” the bearing into small 
spots too soon, creating a “salt-and-pepper” 
effect, but with wide gaps between the 
spots. A poorly done job will sometimes 
reveal either the remnants of spots in a 
straight row, which can only mean they 
are retained from the original planing, or 
telltale hard spots on the edge of the plate 
where the spots are harder to hit. A require¬ 
ment of a certain number of spots to the 
inch does not mean as much as would be 
supposed. For instance, a surface plate 
worn evenly may have wide spots and still 
be flat. Of much more importance is the 
uniformity of bearing. 

The procedure just outlined will result 
in three plates flat to 50-75 millionths of an 
inch [one or two (im|. They are not yet truly 
flat due to their own elasticity, despite 
adequate ribbing and support. The bearing 
may appear perfect, but the plates contin¬ 
ually bend to conform one to the other. 

To achieve the sought-after 25 millionths 
of an inch [0.0006 mm] requires inspection 
at this point. A two-footed twist gage, Fig. 
19, is employed. An electronic indicator, 
with its base resting on the plate, is set so 
that its measuring probe registers zero or 
datum on a small central lapped pad on the 
gage, when the end feet rest on the diagonal 
corners of the plate. Leaving the zero set, 
the twist gage is quickly swung 90° to the 
opposite diagonal corners. The relative 
difference in heights, or “twist,” is regis¬ 
tered as a plus-or-minus reading against 





fig. 20 -Three-footed twist gage has two 
gaging pads at the corners of one side. A 
smalt gage having a slight taper of 
0.000050 inch per one-half inch 
[i approximately one pm per 10 mm] 


of length is applied at the two 
gaging pads. The twist gage is self-proved 
upon being turned 90° on the surface 
plate and tested once again. 


the lapped pad. Even though the twist 
gage is known to sag appreciably, this sag 
is a constant, as evidenced by extremely 
good repeatability of the zero. 

More local errors may be detected by 
exploration with the three-footed twist 
gage, Fig. 20. The gaging points with this 
instrument are two lapped flats, fixed on 
the bottom corners of the single-foot side, 
and facing the plate. A wedge-shaped gage, 
tapered accurately 0.0003 inch over 3 inches 
[0.0076 over 76 mm] or 0.000050 inch for 
every inch of length [0.00254 mm for every 
25.4 mm] of length, is pushed under the 
lapped flats to calibrate change in height of 
the two lapped flats. 

A third method uses a surface gage with 
a protruding arm to which an electronic 
indicator probe is fixed. The plate may be 
searched for even more local errors in this 
manner, Fig. 21. 

It is important to note, however, that if 
the correct scraping procedure is followed, 
errors will not be of a local nature, but only 
those of overall bow or twist. 

In a fourth method, overall “bow” is in¬ 
spected with a “straddle” surface gage, and 
a special straightedge having two support 
feet .554L apart* and three indicating pads, 
Fig. 22. The straightedge can be reversed 
for self-check, Fig. 23. 

There will be a large area in the center of 
the plate which is perfectly flat. The ex¬ 
planation for this will be seen in a test for 
rigidity: 

The two-footed twist gage is positioned 
on diagonal corners and the indicator 
zeroed on the central pad. Two hundred 
and fifty pounds [113 kg] can be added to 
the center of the table with a deflection 
of no more than 5 millionths of an inch 
[0.000127 mm]. Two hundred and fifty 
pounds [113 kg] added to the corners will 


*When a straightedge is supported on two points 
equidistant from its ends and separated by the factor 
.554 L (where L is the length of the straightedge), it will 
deflect the least from elastic flexure. 




FIG. 21 —In searching for local errors, an 
electronic indicator and stand and a small 
lapped block are employed. 
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fig. 22 —Overall curvature of the plate is 
inspected with a “straddle” surface gage 

and a straightedge having support feet fig. 23 —( insert ) The special straightedge is 

,55fL apart, and three indicating pads. self-checked by reversal. 



cause a deflection of 15 millionths of an 
inch [0.0004 mm]. 

The plates must now be scraped not only 
to the bearing, but to the errors found from 
inspection. There is no doubt that the 
rouge is slightly abrasive and to a certain 
extent also laps the plates flat, but the 
practice of stoning to remove local errors is 
of dubious value. The temporary accuracy 
achieved is at the expense of uneven wear 
as the plate is used. Stoning may be used 
sparingly and evenly over the whole plate 
but only for the purpose of removing the 
slight burr resulting from scraping. 

An autocollimator may be used to cali¬ 
brate a master plate, using the method 
shown diagrammatically, Fig. 24. While it 
is an extremely sensitive instrument, one 
disadvantage is greater time consumption 
(during which temperature fluctuations 
may change the datum). Cumulative errors 
may result from moving the reflecting 
target mirror and changing the span of the 
target mirror feet. The necessity of making 
charts and analyzing the results can con¬ 
tribute to error. Accuracy of determination 
is not much better than the 50-75 mil¬ 
lionths of an inch [one or two |um] achieved 
without an inspection. 

15-Minute Check for Flatness 

This dramatic test was made on three 42 
inch [1067 mm] master plates— 

Three plates were matched one against 
the other for bearing. Bearing was close 
and uniform. 

The two-footed twist gage was set on 
diagonal corners on the first plate and a 
datum zero set. Moving the twist gage to 
the opposite corner, then actually repeat¬ 
ing this on the other two plates without 
changing the datum (this involves care¬ 
fully moving the electronic indicator and 
stand), revealed no more than 15 mil¬ 
lionths of an inch [0.0004 mm] spread of 
indicator readings for all 6 diagonals. Total 
time elapsed: 15 minutes. 


A guest metrologist who was present at 
this time conceded that autocollimating a 
surface plate completely, as illustrated in 
Fig. 24, would take about three hours per 
plate, or a total of nine hours. 

If the scraping were not of the highest order, 
so that the “bearing” could be relied upon to 
prove the absence of local error, the 15-minute 
check for flatness would be neither adequate 
nor valid. 

Cast iron is of fairly porous texture. Can 
such a surface, which is, moreover, flat 
only on “plateaus” spaced apart with de¬ 
pressions, be called flat to 15 millionths of 
an inch [0.0004 mm]? 

Such tolerances are stated with the reali¬ 
zation that they begin to conflict with the 


controversial subject of “surface texture” 
and all its inherent uncertainties and dis¬ 
crepancies (see pages 125-129). On the other 
hand, such readings are entirely reliable 
when a square inch [25 mm square] or so 
of the scraped surface can be spanned, 
Fig. 25, Fig. 26. Further clarification is 
difficult since “surface texture” definitions 
are not adequately descriptive. 

Granite Surface Plate 

Granite, for use as a surface plate, is ac¬ 
knowledged to be a very satisfactory ma¬ 
terial for all types of shop, inspection and 
laboratory work. It is by far the most 
widely used. 

On the other hand, the claim that the 
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FIG. 24—77ie calibration of a master 
surface plate (as shown with an 
autocollimator ) requires about three hours 
and involves cumulative errors. 


fig. 25 ( center)—A scraped surface is made 
up of a scries of plateaus. To indicate 
directly over this surface is incorrect 
procedure. Instead, a sufficient area should 
be spanned with a hardened steel block, 
which has been lapped flat and parallel. 


cast iron surface plate is headed for extinc¬ 
tion is not borne out in actual practice. 
Many machine-tool manufacturers, for ex¬ 
ample, still use and prefer cast iron surface 
plates. 

The granite surface plate does not pro¬ 
vide as acceptable a “bearing” as does cast 
iron to which one can scrape masters and 
machine members. For this reason, it does 
not have a place here in the “evolution” 
which will be followed to machine geom¬ 
etry. 

See outline of the most important crite¬ 
ria to apply in closer evaluations of the ad¬ 
vantages and disadvantages of cast iron 
compared to granite for use as a surface 
plate. One point in the outline calls for 
further clarification: 

Granite and Moisture— The claim that 
the flatness of a granite surface plate is 
effected by moisture has not been proved 
conclusively. 

In a controlled test by Oakley*, two black 
granite plates were copiously wetted with 
water for some 70 hours. Measurements 
made at intervals after the surfaces of the 
plates had been dried showed only a small 
change in flatness, but revealed that the 
plates took about 10 days to regain their 
original flatness. This would seem to argue 
against the practice of wetting to clean. 

Oakley cautions, however, that the 
effect of a high-humidity atmosphere has 
not been tested. This raises the possibility 
that some loss of flatness may result from 
high humidity.** 

The results should also alert manufac¬ 
turers of granite surface plates to the po¬ 
tential danger involved in lapping with a 
liquid solvent. The prevalent practice of 
lapping followed by immediate autocol- 
limation should be avoided. 

*T. R. J. Oakley, “The effect of moisture on the 
flatness of black granite surface plates,” “NPL Tech¬ 
nical Bulletin GGG-P-463b, August 3, 1961. 

♦♦Unpublished letter from T. R. J. Oakley, Febru¬ 
ary 21, 1968. 







FIG. 26— “Scraper’s block” of hardened 
nitralloy steel, lapped flat and parallel 
to .000005 inch [0.000127 mm], is used 
as a gaging point when inspecting scraped 
surfaces. 
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A Comparison Between Cast Iron and Granite 
for Use As Surface Plates 


CHARACTERISTIC 

Wear 

Stability 

Rigidity 

Relative Weight for Comparable Strength 

Moisture 

Fabrication, reconditioning 
Bearing 

Accuracy 

Burr 

Available shapes and sizes 
Versatility in use 

Temperature effect (absorption) 

Temperature effect (radiation) 

Temperature effect (coefficient) 

Cost 


CAST IRON 

Good 

Excellent 

Excellent 

High strength for given weight 
(ribbing easily cast integral to plate) 

Rusts, but will not distort 

Lapped or scraped 

Will give up a bearing (advantage) 

Depends on manufacturer 

Raises up (disadvantage) 

Limited, often needs special pattern 

Excellent—can be machined, 
drilled and tapped 

Fast to absorb heat, slow to dissipate 
(can be advantage or disadvantage) 

Reflects radiant heat (advantage) 

The same coefficient of expansion 
as most steels (usually an advantage but 
can be a disadvantage) 

Fairly expensive 


GRANITE 

Good 

Excellent 

Excellent 

Strong, but less for given weight 
(impractical to apply ribbing) 

Won’t rust, but may distort 

Must be lapped 

Will not easily give up a bearing 
(disadvantage in use, may also limit 
attainable accuracy) 

Depends on manufacturer 

Makes a hole (advantage) 

Almost unlimited, can be sawed or shaped. 

Good, but needs inserts and fasteners. 

Slow to absorb heat, slow to dissipate 
(can be advantage or disadvantage) 

Absorbs radiant heat, 

especially black granite (disadvantage) 

Coefficient unlike steel 
(usually a disadvantage 
but can be an advantage) 

Relatively inexpensive 
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fig. 27 —A straightedge of the form shown 
was used to scrape the dovetail ways of 
No. 1 and No. 2 Model Jig Borers, TIs 
accuracy was limited due to lack of 
rigidity and because Us ff> m form ivas not 
self-proving. 


2. STRAIGHTEDGES 

All straightedges are derived from the flat 
plane, but design will vary with different 
applications. 

A straightedge, Fig. 27, was used as a 
master for scraping the dovetail ways of the 
Moore Model No. 1 and No. 2 machines. 
Its uniform structure minimized distortion 
from heat, but it was not sufficiently rigid 
to prevent bending, either from its own 
weight or from the application of pressure 
to produce a bearing. A further disadvan¬ 
tage was that its 45° form, unlike a 90° 
form, was not selfproving. 

In the quest for a straightedge capable 
of producing closer accuracies, other de¬ 
signs evolved, Fig. 28. It was observed that 
while each would resist bending in one di¬ 
rection, none possessed sufficient cross- 
section rigidity to resist bending in all 
directions. Also, being of non-uniform con¬ 
struction, they easily distorted from un¬ 
even temperature distribution. 

Case History No. 1 

A straightedge of a type shown at left in 
Fig. 28 was matched to a master surface 
plate. The bearing pattern continually 
shifted, yet tests indicated that the shift 
was not caused by deflection. A chance re¬ 
mark revealed that the surface plate had 
recently been transferred from another area. 

The suspicion that temperature was the 
culprit was easily confirmed by deliberately 
holding the top of the straightedge for a 
short period of time. The heat of the hand 
made the bearing pattern shift even more 
drastically. What had taken place was that 
the temperature of the air differed from the 
temperature of the surface plate. The top 
of the straightedge was influenced by air 
temperature, and the base was influenced 
by the temperature of the surface plate 
upon which it rested, with uncertain gra¬ 
dients of temperature in between. The re¬ 
sult was a changing and unpredictable dis- 






FIG. 28— The straightedges shown rvere at 
various times used as masters for 
machine ways, hut were not self-proving, 
not sufficiently rigid, and unduly sensitive 
to temperature variations. 








FIG. 29 — The box straightedge: is ihe best 
design. It is extremely rigid, its 90° 
form is self-proving and it is relatively 
insensitive to temperature change. 


tortion of the straightedge due to its own 
non-uniform temperature. 

Conclusion— This type of straightedge 
was judged too susceptible to distortion un¬ 
der slight temperature differentials, making 
it impractical to use where close tolerances 
are required. 

Case History No. 2 

A machine V-way was scraped, using 
the middle straightedge of Fig. 28 as a 
master, until the bearing was uniform. To 
self-prove the straightedge it was turned 
end-for-end in the V-way. The bearing was 
still uniform. Yet subsequent autocollima- 
tion revealed that machine “V” had con¬ 
siderable horizontal deviation from a 
straight line. 

Conclusion— This straightedge was bend¬ 
ing to conform to the machine “V,” and 
had insufficient rigidity to be used as a 
master. 

Bearing in mind the limitations which 
were apparent with conventional straight¬ 
edges, a master straightedge was designed, 
Fig. 29, with the following features: 

1. Rigid cross section to minimize de¬ 
flection in all directions; 

2. A symmetry and uniformity which 
minimize distortion when the straight¬ 
edge is subjected to uneven tempera¬ 
ture distribution; 

3. A 90° form to be used as a master to 
construct a master “V”; 

4. A 90° form and straightness which are 
self-proving both in fabrication and 
in use. 

Creating a Master Straightedge 

Fig. 30 shows the approach taken to scrape 
the master straightedge. Side 1 of the 
straightedge is matched to the master sur- 



Side No. 2 
(Flat and parallel 



FIG. 30 —The sides of the box straightedge 
are scraped in the order of ihe numbers 
listed. 
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fig. 31 —After side 1 is scraped flat, side 
2 is scraped parallel to it. Inspection uses 
the master surface plate as a flat datum. 



face plate. Side 2 is also matched and, by 
means of a thickness check, Fig. 31, also is 
scraped parallel to side 1. Side 3 is scraped 
square to sides 1 and 2. Inspection is by a 
common method, Fig. 32. With the straight¬ 
edge supported on side 1, a knife edge on 
the indicator stand is pressed against side 
3, and a zero set. The straightedge is turned 
180° to side 2, the indicator and edge again 
pressed into contact with side 3. 

Any out-of-squareness is revealed as half 
the difference between the two indicator 
readings and gives the direction in which 
side 3 must be adjusted for squareness while 
still maintaining flatness. For highest ac¬ 
curacy and in order to avoid local flatness 
errors in the master surface plate, the 
straightedge should be turned end-for-end, 
as well as through 180°, so that the position 
of the indicator stand on the surface plate 
is not altered. 

Once 3 is completed, side 4 is scraped 
parallel to it, which means that side 4 is 
automatically square to sides 1 and 2. 

The resulting straightness can be no 
better than the 25 millionths of an inch 
[0.0006 mm] flatness of the master surface 
plate, since the straightedge will bend this 
amount and, in fact, may have as much 
as 40 millionths of an inch [0.001 mm] bow. 
This is not revealed in a check either for 
bearing or for thickness. Any improvement 
upon the 40 millionths of an inch [0.001 
mm] straightness can only be achieved by 
proving the straightedge against a Measur¬ 
ing Machine. Squareness and parallelism, 
however, are obtained to 10 millionths of an 
inch [0.00025 mm], since the overall bend¬ 
ing does not effect local cross-sections. 

It is important to note that both straight¬ 
ness and squareness have been developed 
from the surface plate. Other designs of 
straightedges and squares are created simi¬ 
larly. 


FIG. 32— Side 3 is scraped flat and square 
to sides 1 and 2. The knife edge on the 
indicator stand is pressed against the 
lower area of side 3. Any out-ofsquareness 
is revealed as half the difference tf the 


indicator readings when the 
straightedge is turned over ( 180 °), to side 
2 and compared again as shown. 
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PIG. 33— A single, female V can be made 
extremely accurate simply by scraping to 
the bearing imparted when rubbed to the 
master straightedge. 


The function of the master straightedge 
will be to produce the straightness and 90° 
form of a master “V.” 

3. THE FEMALE DOUBLE V 
MASTER 

An accurate female V master, Fig. 33, is 
created by matching a machined V to the 
right-angle corner of the master straight¬ 
edge. Purely by scraping to the bearing im¬ 
parted, and without the use of auxiliary 
inspection devices, the 90° form of the V 
will be in error less than 10 millionths of an 
inch over its 1 Y 2 inch depth of flank [0.00025 
mm over 38 mm] and its straightness will 
be close to 0.0001 inch [0.0025 mm] accu¬ 
racy over its length, for these reasons: 

1. The straightedge is highly resistant 
to deflection across its diagonal sec¬ 
tion; 

2. Since the squareness error of the 
straightedge over 8 inches [203 mm] 
was less than 10 millionths of an inch 
[0.00025 mm], the 1 finches [38 mm] 
of corner being used will be propor¬ 
tionately less in error. 

3. The straightedge can be self-checked 
by reversal; 

4. Each of the four corners of the 
straightedge provides another cross¬ 
check and long wear-life. 

The ability to achieve straightness to 25 
millionths of an inch [0.0006 mm] depends 
on the support given the female master. If 
supported only on a rough-cast surface 
when being rubbed for bearing, it will bend. 
Even the precaution of supporting the fe¬ 
male master on its points of least deflection 
will not prevent bending when the weight 
of the straightedge is added, Fig. 34. The 
proper method is to scrape the underside 
of the female master flat, and when being 




fig. 34 —The female double V master will 
easily bend if supported on a rough-cast 
surface, or even if supported on the points 
of least deflection, as shown. 
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rubbed for a bearing, to support it on the 
master surface plate, directly over the 
triangle of the three-point support, Fig. 35. 

The single V must be straight and of 90° 
form. The double V presents, in addition, 
more complex problems of geometry: lean, 
center-distance, horizontal parallelism and 
vertical parallelism (“twist”), Fig. 36. For 
reasons of machine design, which will be 
fully explained in a later section, Moore 
uses double V-ways of 6)4 inch [165 mm] 
and 8 J ><2 inch [216 mm] center-distance 
on the base construction of its No. 1}/$ 
and No. 3 machines. The female double V 
master will be the authority for these 
double V’s. 

Scraping each V to the straightedge only 
results in two straight 90° forms, but ran¬ 
dom in space. To bring the V’s into a pre¬ 
scribed geometric relationship to each 
other, there must be a systematic inspec¬ 
tion and correction of each type of error 
mentioned in Fig. 36. 

The now-flat underside of the female 
master not only provides a deflection-free 
support, but in conjunction with the 
straightedge and the 48-inch [1219 mm] 
plate on which it is supported, provides a 
reference datum in many of the following 
inspection procedures. 

1. Form — Fig. 37 .—The 90° V is chosen 
mostly because of its unique relationship 
to the flat plane, so it is easily duplicated 
and proved. An exact 90° is not a pre¬ 
requisite to the ultimate accuracy of the 
machine V provided the mate to the 
female V is of an identical angle. 

It has been seen that the right angle is 
precisely achieved through use of the 
straightedge. One method of proof of the 
90° angle sometimes suggested in treatises 
on measurement is by means of two round 
pins, such as 2)4 inches [57.15 mm] and 





























FIG, 37— The 90° V-forrn is established by 
the master straightedge and can be made 
very accurately because of its unique 
relationship to the flat plane. 



inch [12.7 mm] whose sizes are accurately 
known, Fig. 38. 

Measuring their relative differences in 
height in the V by comparison with gage 
blocks will reveal the correctness of the 90° 
angle. This method, however, involves 
many cumulative errors (size and round¬ 
ness of the pins, picking up their high point 
in the V, the line-contact of the pins in the 
V, the error in the gage blocks, etc.). A better 
method would be to use a master cylinder 
square in the V and view a light gap against 
one flank of the V while set against the 
other. 

Neither method would match the accu¬ 
racy already achieved by matching the V to 
the straightedge. 

2. Lean — Fig. 39 .—In two methods of in¬ 
specting for “lean,” the V’s are compared 
to a common datum, the flat underside of 
the female master. In the first method, a 
4 inch x 4 inch [100 mm x 100 mm] square 
gage, fixed to the end of the straightedge 
which rests in one of the V’s, is inspected 
for parallelism to the surface plate, Fig. 40. 

Lean of the V relative to the base of the 
female master shows as an out-of-parallel- 
ism of the gage. Lean of the V’s relative to 
each other shows as an out-of-parallelism 
of the gage when the straightedge is placed 
in the second V. This set-up is self-proving. 
If the straightedge is truly square, paral¬ 
lelism of the gage will be identical as each 
of the four corners of the straightedge are 
placed in the same V. The straightedge 
may be turned end-for-end to inspect the 
square gage. 

In another method,, a small V-gage, 
which has two sides at 90° to each other, is 
used. The third side, at 45° to the other 
two, is indicated across, Fig. 41, to discover 
any localized lean. Repeat readings made 
when the gage is reversed in the V-way 
self-checks the gage. 



= 2.112437 


All dimensions shown are in inches. 


FIG. 38— One method of proof of the 90° 
form which is sometimes advocated does not 
result in accuracy in practice. 
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FIG. 4 0—Lean of (he V is inspected as 
shttvn. The straightedge is self-proved 
by being t urned end-for-end as well as 
by placing each of its corners in the V 
FIG. 39- -Lean, and repeating the check. 














accurate performance , providing mating 
members are the same center-distance. 


FIG. 4 A —(center) Measurement of center- 
distance of the Vs by means of gage pins is 

not sufficiently reliable. 


FIG, 42. Center-distance of the V's. 

Standardized center-distances of 6.5000 
inches [165.100 mm j and 8.5000 inches 
[215.901) mini are used for convenience of in¬ 
terchangeability of mach ine-me mbers. An 
exact center-distance is not a pro requisite to 



3. Center-Distance. Fig. 82. — It might be 
debated that the center-distance of two 
V’s is not a derivative of the flat plane but 
a linear function, hence more properly 
treated under “standards of length.” 
However, as with the 90° angle, a known, 
measured center-distance is not a pre¬ 
requisite to an accurate system. Mating 
V’s must only be compared by inspection 
to verify that they are of the same center- 
distance. 

Standardized center-distances of 6.5000 
inches [165.100 mm] and 8.5000 inches 
[215.900 mm] are used on No. 3 and No. 1 Vz 
machine V-ways for convenience and 
interchangeability.* 

In one method of establishing distance, 
the correct stack of gage blocks must fit 
between two pins in the V’s, Fig. 43. Since 
this method relies too much on “feel,” a 
better method is to check the center- 
distance of the V’s on the Moore Univer¬ 
sal Measuring Machine, Fig. 44. 

Once the center-distance is established, 
it is transferred to other “working” 
masters by means of a special double-V 
gage. One V of this gage is reed-hinged to 
allow setting against an electronic in¬ 
dicator, Fig. 45. Once a datum or “zero” is 
set, the scraping master may be com¬ 
pared to the double-V gage, Fig. 46. The 
female master is never again used in pro¬ 
duction, but only periodically as the 
ultimate arbiter of center-distance. 

4. Vertical and Horizontal Straightness, 
Fig. 87 (left and right respectively). It has 
been seen that 0.0001 inch [0.0025 mm] 
straightness can be achieved through use 
of the straightedge alone, but straight¬ 
ness in the horizontal plane is more easily 
achieved than that of the vertical. The 
reason is that the straightedge,when rest- 


*Due to historical circumstances, the standardized 
center distances are not actually nominal as implied, but 
are known precisely. 


fig. 44 -The Universal Measuring Machine Chief Inspector here establishes 

is the. most accurate method for establishing center-distance of the ultimate master, 

center-distance of double V masters. which is a hardened, ground and lapped 

double V gage. 
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I 1 'vc. 45--To transfer center-distance from 
the ultimate master, a double V male gage 
is used. One V of the gage is reed-hinged 
to allow setting a datum with an electronic 
indicator. 


fig. 40 —Once the datum is set, the 
center-distance of the female double V 
master ran he compared to the hardened 
master and scraped to agreement. 



































FIG, 48 — The straightedge imparts its 
straigh tness to the female V, The 
straightedge is self-proved upon being 
reversed 1.80° and examining the bearing. 
'The method is sensitive, since any errors 
are opposed upon reversed. 



ing in the V-way, has greater force down 
through its vertical diagonal due to its own 
weight than across its horizontal diagonal. 
The straightedge thus bends more to con¬ 
form to errors of curvature of the female 
master in the vertical plane. 

Self-proving the horizontal by revers¬ 
ing the straightedge end-for-end reveals, 
through bearing change, even the slightest 
bow of the straightedge itself, Fig. 48. Self¬ 
proving the vertical direction by turning 
the straightedge upside down, Fig. 49, self- 
proves the vertical straightness, but to 
some extent repeats the vertical deflection. 
Fortunately, the vertical straightness devi¬ 


ation can be given two good inspections. In 
the first, change in height of a lj/g inch x 
iy 8 inch [28.6 mm x 28.6 mm] square section 
gage with a small indicating pad is recorded 
over the length of each V. In a second 
method, each V is autocollimated for verti¬ 
cal straightness, Fig. 50. 

The most convenient method of inspect¬ 
ing horizontal straightness of each V is by 
autocollimation. 

5. Horizontal Parallelism, Fig. 51. Since 
all double-V’s at Moore have standard 
center-distances, ultimately derived from 
the 6 J /2 inch [165 mm] and 8 J/£ inch [216 


mm] female double-V masters, the actual 
center-distance need no longer be measured. 
The problem is now only that of matching of 
the V’s in terms of horizontal parallelism, 
without the connotation of any actual 
distance. 

Horizontal parallelism is inspected with 
a fixture, shown in Fig. 52 (or the gaging 
setup of Fig. 45 may be used). An indicator 
and arm are fixed to a cylinder resting in 
one of the V’s. Rotating the cylinder slowly 
in the V (against a spring helper) registers 
a zero of the indicator against the high 
point of a cylinder in the adjacent V. 
Parallelism is charted by comparing sub- 
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fig. 40 - Vertical straightness is 
self-proved by turning the straightedge 
upside down. The indicator shown at the 
left is used to verify the straightedge, as in 
Fig. fO. 
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sequent readings taken over the length of 
the V’s to this zero. The latter inspection, 
while it may reveal certain errors of straight¬ 
ness, should not be the basis of correction 
for straightness. (Attention is called to 
“Requirements of an Inspection Instru¬ 
ment,” page 19). This inspection does not 
reveal which V deviates, or how much. Sim¬ 
ilarly, an inspection for horizontal parallel¬ 
ism would discover no error in two V’s which 
were parallel but not straight. 

6. Vertical Parallelism—known also as 
“twist", Fig. 53. “Twist” of the V’s is 
inspected in the same manner as the indi¬ 
cator method under “verticalstraightness,” 
Fig. 54, using a precision plug gage, except 
that the V’s are now presumed to be 
straight, and are inspected for parallelism. 

Autocollimation will only reveal devia¬ 
tion from a straight line, and thus does not 
inspect either horizontal parallelism or 
twist. 

For simplicity, the procedure outlined 
was confined to the female master. In 
practice, a mating male double-V master 
was developed simultaneously by matching 
it to the female master. By reversing, both 
the male and the female masters are self- 
checked for center-distance and straight¬ 
ness, Fig. 55. Although accurate, the 
straightedge is not as sensitive and reliable 
as is the final check where mating double-V 
masters are reversed. It is a definite advan¬ 
tage to have masters that encompass both 
rails. 

Of all the various way-designs in com¬ 
mon usage, such as the V and flat, or the 
gib-type, only the double-V can be reversed 
as a self-check, if a master is to be used 
that gives bearing on both rails at once. 

The inherent value of the double-V con¬ 
struction was apparent as early as 1931 
when it was used between the column and 
housing of the No. 1 machines, Fig. 56. 
The double-V design was repeated in the 
column of the No. 2 machines. This ex- 







pig. 51 —Horizontal parallelism of the 
T’s must be generated and inspected token 
a double V-way design is employed. 


HORIZONTAL PARALLELISM 


FIG. S3 —Vertical "parallelism or “twist 
is one of the most difficult and 
troublesome errors to eliminate in machine 
slide-ways. Many machine-tools hare a 
twist in their ways bemuse the masters 
used are developed from surface plates 
haring a “twist.” 


VERTICAL PARALLELISM OR “TWIST" 


pig. 52 —Horizontal parallelism is 
inspected with this gaging set-up, or by the 
reed-hinged gage shown in Fig. f6. 
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fig. 54 — To inspect for tivist, the height of 
both V’s is compared over full length, using 
the master surface plate as the datum. 


fig. 56— The advantage of the double V 
was recognized early and was used in the 
column of the No. 1 Jig Borer. 








perience formed a basis for the evolution 
of skills and techniques necessary in work¬ 
ing to a double-V. 

In singling out the specific types of 
errors, the writer recognizes that it is one 
thing to discover them and quite another 
to translate analysis into correction. Scrap¬ 
ing the double-V, involving many combina¬ 
tions of errors, will put a good scraper to 
his mettle. He must thoroughly understand 
the “geometry” involved, for he may be 
scraping for several types of errors simul¬ 
taneously. An unnecessary amount of time 
may be expended if he fails to put a proper 
value on each. 

Usually, several years of training are re¬ 
quired before a man is given the task of 
scraping a double-V. It is readily admitted 
that when badly done, a double-V can be 
the worst type of machine construction. 
An accurate double-V, on the other hand, 
besides being a “natural” standard, will be 
seen to be the most advantageous in terms 
of actual machine performance. 

4. MACHINE DESIGN 

Coordinate Location 
A machine tool with two movable axes, 
“X” and “Y,” each with its measuring 
element, can position to any point in a 
plane, Fig. 57. X and Y axes may both 
carry the piece to position it relative to the 
spindle, or the X axis may alone carry the 
piece and the Y axis carry the spindle. 
Coordinate location may be in a vertical as 
well as a horizontal plane. 

With a third axis, “Z,” perpendicular to 
X and Y, the location is to any point in a 
cube bounded by these three travels. A 
full measuring element is not always neces¬ 
sary in the Z axis (a machine may work 
mainly on through holes), but auxiliary 
means are usually included to establish 
distance. 


fig. 55 —The advantage of the double V is V masters, or machine-members to be 

best illustrated here. If male and female reversed provides a quick, sensitive check 

conform, but are not straight {left), then which is alivays available, 

upon reversal [right), the errors are 
opposed. The bearing is non-uniform 
(lower right). The ability of double 
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fig. (a—The, theory of coordinate-location 
to estoMish any point in a plane is shown 
here. The principle is the same with all 
coordinate-locating machine-tools, even 
though the type of machine construction, 
way design and measuring element may 
vary. 


Geometry as a Function of 
Machine Accuracy 

Straightline Travel: A Factor in 
Locational Accuracy 

Relatively slight errors of straightness in a 
machine’s ways may result in very sizable 
errors in location when machining or mea¬ 
suring holes and surfaces in an actual work- 
piece. 

The nature of this error can be pictured 
as spokes in a wheel, Fig. 58. For example, 
a horizontal curvature of only 0.0001 inch 
in 10 inches of travel [0.0025 mm in 250 mm] 
will introduce a locational error of 0.0008 
inch at 10 inches out [0.02 mm at 254 
mm] from the machine’s lead screw (or 
other measuring element). The actual 
travel is greater or less than nominal de¬ 
pending on whether the interval that is set 
is on the side of the lead screw axis farther 
from or nearer to the center of curvature of 
the ways. 

The same magnification of error also 
occurs in the vertical plane. In the case of 
a concave arc, Fig. 59, travel is foreshort¬ 
ened, the axes of the holes are out of paral¬ 
lel, and point at one another at some dis¬ 
tance above the table. 

In the case of a convex arc, Fig. 60, 
travel is lengthened, the axes of the holes 
are out of parallel, and point at each other 
at some distance below the table. 

The principles involved here have to do 
with what is often referred to in metrology 
as the “Abbe effect.” 

Causes of an Arc in Travel 

The causes of an arc in travel are: 

1. Original error of straightness scraped 
or machined into the ways. 

2. Deflection of machine members. 

To prevent deflection, each axis must be 
supported in its full travel. The designer is 
often tempted to increase the travel of the 
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All dimensions shown are in inches. 


fig. fig —Curvature of travel is best 
visualized as spokes in a wheel. The 
measuring element remains unaltered, but 
the actual distance the part moves varies 
along the spoke. This error is very severe 


and occurs regardless of the type of 
measuring element in use (lead screw, 
precision scale, laser interferometer, etc.) 
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fig, 59 —A concave travel in the vertical 
plane is shown. 0.0001 inch curvature in 
10 inches of travel [0.0025 mm per 250 mm] 
will cause a shortening of travel of .0008 
inches [0.02 mm] at a distance 10 inches 
[250 mm] above the ways. 


Only 9.9992 inches travel at 
10 inches above ways. 

(In error by 0.0008 in.) 




axes without support, since it is less expen¬ 
sive. The tolerances required of vertical 
millers and similar machines will permit 
this, but it is an unfortunate compromise 
in high-accuracy machines. The resulting 
“overhang” causes a sag as the table passes 
beyond its support, Fig. 61. Deflection 
from weight can have a constant value, but 
the conditions of deflection must be iden¬ 
tically repeated for this to be so. For ex¬ 
ample, in Fig. 62, the ways can be scraped 
concave to compensate for the deflection 
caused by the table, but not for the variety 
of weights of workpieces that may be 
mounted on it. 

Wear of Machine Ways 

The ways in a machine tool tend to wear 
concave, because the center of the travels 


is most used. Wear can be minimized in de¬ 
sign by: 

1. An overall machine configuration or 
way design that tends to minimize 
wear; 

2. Having the two guideways on each 
axis subject to uniform load, uniform 
friction; 

3. Preventing excessive wear on one 
short portion of the travel. 

The machine’s measuring element is usu¬ 
ally blamed for loss of positioning accuracy 
when more often the cause is wear in the 
machine’s ways. 

Deliberately putting an error into the 
measuring element to compensate for wear 
is a poor practice. Accuracy is still attained 
at one distance and only one distance from 
the ways, Fig. 63. At all other distances 
along the “spoke,” that axis will be in error. 
Instead, every possible design precaution 
should be taken to discourage wear, espe¬ 
cially uneven wear, since even a relatively 
small arc can produce serious errors. 

Another poor substitute for geometric 
accuracy is shifting a machine tool’s pre¬ 
cision scale to compensate for curvature of 
travel. 

Squareness of Travel Is a Factor 
in Locational Accuracy 

Coordinate location demands a 90° rela¬ 
tionship of the X-Y travels. Out-of-square¬ 
ness causes a locational error, Fig. 64. Out- 
of-squareness most often has a constant 
value with each machine, but there are 
circumstances under which this is not so. 
Squareness may be altered by deflection, 
shift of the travels (due to poor guide 
ways), or heat. If the axes do not travel 
straight, squareness calibration can only 
have a mean value, Fig. 65. 

When used in one relative position, the 
Z axis is usually not a function of coordi¬ 
nate location accuracy. However, loca- 


fig. oo— A convex arc in the vertical plane 
is illustrated. Error is magnified as in 
Fig. 59, but the center-distance of holes is 
now established too far apart, and out of 
parallel as shown. 
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fig. 62 —Attempts to compensate for 

FIG. fil —If the axes are not supported over “overhang” by scraping the ends of the 
full travel, movement of machine members travels high (left) is inadequate, since the 

causes a sag, resulting in errors as shown deflecting force is not always the same 

in Figs. 59 and 60. (right). 



Measuring element compensated for height “h”. 


Actual inch 



Travel 





fig. 63 —Compensation for curvature of ftg. 6-1 — Out-of-squareness of X and 1 ’ fig. 65— Jf the axes do not travel straight, 

travel by lengthening the measuring element, axes causes a locational error. squareness can only have a mean value. 

or shifting it while the axis moves, is also a 

poor substitute for geometric accuracy. The 

actual travel is only correct at one point 

along the “spoke.” 
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fig. m—Locational error may be caused by 

the out-of-squareness of the Z axis, when 
boring holes al different heights. 



Error 


Z axis 

4 


Column 



tional error is in proportion to the out-of¬ 
squareness of the Z axis, and the extent of 
the Z axis used, Fig. 66. 

An additional error is caused when quill 
travel is not parallel to the Z axis, Fig. 67. 

Temperature Effects on Geometry 
of the Machine 

Sources of uneven temperature distribution 
from the environment have already been 
mentioned in regard to scraping masters. 
Consequences of temperature variation are 
even more detrimental to the accuracy of 
a machine tool. 

Most subtle and most difficult to combat 
is “stratification” of room temperature, 
which usually takes the form of a cold floor 
and warm ceiling. A machine tool also has 
its own heat-producers—motors, spindle 
bearings, gear and belt drives, lights, heat 
from cutting. 



It is impossible to completely eliminate 
uneven temperature distribution in a ma¬ 
chine. It is possible to minimize the loss of 
accuracy due to “temperature” through 
observance of the following design con¬ 
siderations: 

1. Sources of heat should be removed or 
insulated from the main body of the 
machine; 

2. Spindle bearing heat should be iso¬ 
lated from the coordinate (X-Y) axes. 

3. If X and Y axes are at the same room 
level, the measuring element of each 
will be more nearly the same tempera¬ 
ture; 

4. Where possible, the coordinate axes, 
the measuring element and the work- 
piece should all be in the same “heat 
sink.” 

MACHINE CONFIGURATION 

Accurate hole-location machine tools are 
best represented by those in the “jig borer 
family” where the most sophisticated con¬ 
struction is employed. Despite individual 
characteristics, jig borers can be classified 
under three basic designs: 

1. Planer-type 

2. Horizontal spindle 

3. Compound-type 

While there is overlapping of function, each 
design has a certain realm of work for 
which it is best suited. It will be observed 
that none of the designs is perfect. A fea¬ 
ture of one may be an advantage only by 
compromising elsewhere. The user must 
analyze the design with a view to what fits 
his own type of work. It is instructive to do 
so with these considerations in mind: 

1. Deflection 

a. Of the machine’s members from 
movement 

b. From cutting pressure 

2. Temperature 

3. Type of workpiece 

4. Operator convenience 

5. Wear 


fig. 07 — Misalignment of the quill may 
also cause a locational error. 
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fig. 68 . The planer-type jin borer achieves 

large capacity in one plane without 
becoming unduly massive, and is best 
suited for flat work. 


The Planer-Type Jig Borer 

The design of the planer-type jig borer, 
Fig. 68, includes a long base which supports 
the table, or X axis, in its full travel. Two 
upright columns, straddling the base, are 
joined at the top for rigidity. The two col¬ 
umns guide and support a cross rail, which, 
in turn, carries the spindle housing contain¬ 
ing the spindle, quill, spindle gears, feed 
gears, etc. The spindle housing, guided by 
ways in the rail, travels as the Y axis. 

The cross rail is positioned vertically by 
two lead screws, one at each end, which must 
be of identical leads to maintain parallel¬ 
ism of the rail to the table at all heights. 

There are two primary advantages of the 
planer-type design. Foremost is the fact 
that X and Y axes can each be given sep¬ 
arate support. This is of benefit especially 
in larger machines to limit deflection in the 
base (contrast this with the compound- 
type). Secondly, it achieves large capacity 
without becoming unduly massive. 

To establish parallelism of the spindle 
housing travel to that of the table, the 
column mounts are scraped. Squaring the 
X and Y travels is accomplished by shift¬ 
ing the columns and then taper-doweling in 
place. 

The rail guide-way is scraped convex to 
compensate for the deflection of the rail 
which results from the weight of the spindle 
housing. The amount of deflection can be a 
constant factor, but only if the conditions 
of deflection are repeated. 

For this reason, it is necessary in order to 
achieve maximum accuracy that the spindle 
housing be returned to a central position 
before the rail is clamped to the columns, 
or is elevated or lowered. This procedure is 
required in terms of the rail-elevating 
screws as well. They must be of an identical 
lead to maintain parallelism of rail to table. 
This is possible only when the load shared 
by each is constant. 

The precaution of coming to center can 
effectively reduce the variance of deflection 


Z axis 



which would otherwise occur, but adds to 
the time required for setting. 

The top way of the rail also is scraped 
convex for reasons of wear. The center of 
travel is subject to more than normal wear, 
since most of the load is carried by the 
upper way in the rail. An arc in this rail 
results in a locational error, magnified with 
the distance of the cutting tool or indicator 
from the ways, Fig. 69. With better condi¬ 
tions of support in the base, there should 
be no undue wear in the base ways. If the 
table travels in an arc in the vertical plane, 
positional error will be magnified with the 
height above the bed ways. 

Worksize limitations are determined by 
the configuration of the machine. The 
piece must fit under the spindle and be- 



FIG. no --An arc in the rail of the 
planet-type jig borer will result in a 
locational error magnified with the distance 
of the, cutting tool or indicator from the 
ways. 
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Fir,. 70 —The horizontal spindle jig borer 

is a rigid design, well-suited for heavy 
machining and box-type workpieces. 


fig. 71 — (center) The quill of the horizontal 
spindle jiff borer sags of its own -weight as it 
is extended from the housing. 




tween the columns. For this reason, the 
planer-type is more suited to flat work. 

Besides the necessity of returning to 
center of balance before elevating the rail 
(which includes the weight of dials, gears, 
etc.), setting time is increased by two other 
factors: 

1. Since the spindle housing moves as 
one of the axes, it is at times difficult 
to reach by the operator for tool 
changing, operating spindle controls, 
or indicating the workpiece; 

2. Separation of X and Y axes places 
positioning indexes and handles far¬ 
ther apart on the machine. 

With the indirect relationship of the trav¬ 
els—the spindle housing travels on the 
rail, the rail on the columns, the column 
fastened to the base—temperature be¬ 
comes more critical. Any localized heat, 
such as from spindle-bearing warmup or 
heat on one of the columns, can more eas¬ 
ily alter parallelism and squareness of the 
travels. Stratifications of room tempera¬ 
ture may cause the measuring element in 
the rail to be warmer than the measuring 
element in the base. 

The Horizontal Spindle Jig Borer 

As with the planer-type, the horizontal 
spindle jig borer, Fig. 70, gives separate 
support to X and Y axes. The bed sup¬ 
ports the table, or Y axis, in its full travel. 
However, the spindle is horizontal, and its 
housing travels as a vertical Y axis. Coor¬ 
dinate locations are then in a vertical plane. 

The spindle housing is usually elevated 
by a lead screw and guided by the ways in 
a single or sometimes double column. 

In addition, there is a Z axis in a hori¬ 
zontal plane on the base, parallel to the axis 
of the spindle, to accommodate different 
size workpieces, or to control depth of cut; 
this axis is equivalent to the rail-height 
adjustment in the planer-type machine. 

A horizontal spindle can sag of its own 
weight as it is extended, Fig. 71, or can 


fig, vz—Use of end supports as shown will The most satisfactory method is to use an 
prevent loss of accuracy of the horizontal accurate indexing table and to in-line bore 

quill only with a great deal of time and care, the piece half-way from each side. 

Use of the Z axis to feed the workpiece 
with the quill fixed is another method 
whereby error due to quill sag is avoided. 
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pig. 73-- The. compound-type jig borer is 
preferred, in the smaller machines. In the 
larger size, it becomes very massive, 
considering its capacity, but is sometimes a 
desirable compromise when the class of work 
falls in between the requirements of the 
planer-type, and horizontal spindle jig borer. 


settle as its weight squeezes out the oil 
film between the quill and its bearing. 

An early method of minimizing spindle 
sag in horizontal spindles by means of end 
supports, Fig. 72, is still widely used today. 
This procedure achieves accuracy only 
with care and time. Another way to cope 
with spindle sag is to use the Z axis to feed 
the workpiece into the spindle. This, in 
turn, introduces location problems, because 
the table rises on an oil film as it moves. 
Use of roller ways will minimize the prob¬ 
lem of oil film, but introduces the even 
more serious problem of shielding the 
rollers against dirt and chips. 

Actually, the problem of spindle sag has 
been satisfactorily solved with the advent 
of accurate indexing tables. To avoid long 
quill extensions, the piece is indexed 180° 
and “in-line” bored from the opposite side. 

Where use of the horizontal spindle pre¬ 
sents problems of sag, there is an advan¬ 
tage in being able to take a heavy cut with¬ 
out deflection: the spindle is backed by the 
full rigidity of the column. 

Extremely heavy workpieces, such as 
box jigs, gear boxes, and housings, can be 
machined most productively on this ma¬ 
chine. They are easily indexed to all four 
sides, a good cut may be taken, and they 
can be accommodated on the table with no 
obstructions. However, flat pieces are more 
awkward to handle because they must be 
mounted on an angle iron to be presented 
to the spindle. 

Operator accessibility to controls is diffi¬ 
cult to achieve in a large machine, and is 
probably best met by the horizontal spindle 
design. The spindle and positioning con¬ 
trols can be reached by the operator from 
one position. 

There should be no inherent wear prob¬ 
lem due to the horizontal design, but an 
arc in the travel would, as in the planer- 
type, magnify positional error the greater 
the distance of the spindle from the column 
ways or from the bed ways. 


With this type of column construction, 
localized heat would cause no undue distor¬ 
tion, but the squareness relationship of the 
travels is not as direct as with the com¬ 
pound-type machine. Spindle warmup can 
be a source of positional error, because the 
spindle housing moves as one of the coor¬ 
dinates in location. With the box-type work 
for which the horizontal machine is best 
suited, the Z axis may be just as important 
as X and Y axes, in which case having a 
spindle in one of the axes is unavoidable. 
Since the spindle housing is at the work 
level, stratification of room temperature is 
not asseriousa problem asin the planer-type. 

The Compound-Type Jig Borer 

The base of the compound-type machine, 
Fig. 73, supports both travels. The com¬ 
pound slide itself, being guided and sup¬ 
ported by the ways in the base, moves as 
the Y axis. The upper ways of the com¬ 
pound slide, guide and support the table 
travel, or X axis. The squareness relation¬ 
ship of the travels (X and Y axes) is thus 
embodied in one member, the compound slide. 

A single column mounted on the back of 
the base carries the spindle housing in its 
vertical adjustment, or Z axis. This ad¬ 
justment is used for depths beyond the 
range of the quill travel, or to accommo¬ 
date larger workpieces, and must be per¬ 
pendicular to the X and Y axes. 

As with the planer-type jig borer, the 
spindle is vertical and coordinates are in 
the horizontal plane. 

Loss of accuracy from temperature is 
minimized in the compound-type machine 
for the following reasons: 

1. The single column design minimizes 
distortion from localized heat; 

2. Table, compound slide, both linear 
standards and workpiece are all in 
the same equalizing “heat sink.” 

3. The linear standards of X and Y axes 
will be the same temperature despite 
stratification of room temperature; 



4. Squareness of the travels, being main¬ 
tained in one member, the compound 
slide, is not likely to be altered by 
temperature conditions; 

5. Spindle bearing heat is isolated from 
the positional axes (although the 
spindle axis will still shift due to 
heat). 

There should be no undue wear due to the 
compound-type design. The most common 
error resulting from wear is a concave arc. 
The workpiece limit is defined by the 
throat distance (column to spindle), and 
by the working height under the spindle 
nose. However, with this design, riser 
blocks (spacers) are easily installed be¬ 
tween column and base to increase work¬ 
ing height. Large, box-type pieces are not 
machined as efficiently as on the horizontal 
spindle machine. 

In the smaller compound-type machines, 
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FIG. 74 — The V and jhit -way design, 
though often considered the ideal, has 
many disadvantages, notably in unequal 
friction of the ways and inability to 
centralize Ike thrust. 



Thrusting element 
cannot be centralized 


positioning handles and spindle controls 
can be reached by the operator from a 
standing, or even sitting, position. Rapid 
adjustment for work height is possible with 
a counter-balanced housing and a simple 
clamping arrangement. The larger the com¬ 
pound-type machine becomes, the less con¬ 
venient it is to operate. 

Because of the small overhang of the 
spindle housing, the advantage of a vertical 
spindle is gained without sacrificing cutting 
rigidity, although relatively speaking, not 
as rigid a cut is possible as in the horizon¬ 
tal-spindle machine. 

Both table and compound slide must be 
supported in their full travels which means 
that, for additional capacity, the machine 
gains in size virtually as its cube. The 
compound slide, in moving, has the addi¬ 
tional weight of the table to support. If the 
table is at the extremes of its travel, or if 
a large workpiece is not centered, the com¬ 
pound slide must move an off-center 
weight. For these reasons, deflection in the 
base and a deflection or shift in travel in 
the compound slide are difficult to prevent 
in the larger compound-type machines. 
Because of lesser workloads involved, this 
is not as much of a problem in the smaller 
compound-type machine. 


fig. 75 — When a weight is placed over the 
flat way of the. V ami flat design, the ability 
of the V to guide is diminished. The 
thrusting element is now not balanced. 



should be here 


In the smaller size machine, the com¬ 
pound-type is to be preferred. The design 
of the larger-size machine depends on the 
type of work it is to do. It is, therefore, 
much more difficult to select a design. 

In the larger machine category, it would 
seem that there is much in favor of the 
horizontal spindle machine where there is to 
be heavy metal-cutting, especially of box- 
type pieces; the planer-type is preferred 
where capacity is needed for large, flat 
pieces, especially measuring-type work; 
and the compound-type as a compromise 
for all-around work. 

Machine Way Design 

So far, it has been shown only how the 
double-V scraping master evolves from the 
flat plane and how its design lends itself to 
being made quite accurately. 

The reason the double-V design is pre¬ 
ferred in a coordinate machine is best un¬ 
derstood by a consideration of the alterna¬ 
tives to it. 

Three classifications, the V and flat, 
the gibbed way, and the central V would 
generally include all those designs most 
commonly used for accurate locational 
machines.* 


FIG. 70 -tignore-ig cm (jibbed way. 
Fundamentally correct, symmetrical. '.I 'hrust 
can be centralized, but once gib is adjusted 
for wear, initial alignment is lost. 


Gib 



Thrust centralized 


Each type should be analyzed in the con¬ 
text of five important criteria: 

1. How good a guide does it provide for 
straightline travel? 

2. What will be the wear conditions? 

3. What type of scraping master can be 
used? 

4. Can the ways be self-checked? 

5. Is a gib necessary? 

1. The V and Flat, Fig. 74 

Despite widespread acceptance and use of 
the V and flat in machine design, it has 
many disadvantages. 

Friction, or “drag” is greater on the V 
than on the flat. 

The single V, placed to one side, is not a 
sufficient guide for horizontal straightness 
when there is side pressure. Thus, the 
thrusting element (lead screw, rack and 
pinion, or other), is not centered between 
V and flat, but is usually placed directly 

*Cylindrical guide-ways have been suggested as 
one other alternative for a jig borer type of machine. 

The shortcomings of this design become evident 
with this question: “How are the cylinders to be 
made and fixed to the machine in such a manner that 
accuracy is achieved, and yet be rigid enough to 
eliminate deflection, either from the sag of the cylin¬ 
ders themselves, or from the weight of a heavy mov¬ 
ing carriage?” 
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vie;. n—Doveia il-type gibbcd way. Resists 
lifting pressures, side shift. 


75 75 75 



under the V, or at least favoring the V, in 
an attempt to balance the thrust, Fig. 74. 
This can only be a compromise, for the 
same conditions of thrust do not apply 
when the workpiece weight is added. More 
seriously, if the workpiece is mounted off- 
center on the flat-way side, Fig. 75, there 
will be a twisting effect, or a “shift” when 
the direction of travel is reversed. This re¬ 
sults because a V-way depends on weight 
to work; if weight is removed, its ability to 
guide is diminished. 

Since there are different conditions of 
friction on each rail, it would not be ex¬ 
pected that this design would wear well or 
uniformly. 

It would be impractical to use a scraping 
master that encompasses both rails since 
the relative height of V and flat would 
have to be maintained so exactly that there 
would be virtually no wear-life to the 
master. If such a master were used, it 
would not be capable of reversal for a self¬ 
check, nor could the table be reversed on 
its bed as a self-check when scraping. No 
gib is necessary with the V and flat design. 

2. The Gibbed-WAY, Fig. 76 and Fig. 77 

The gib-type way, two forms of which are 
shown, actually has much in its favor. 


tug. 78 —The central V-way design. Weight 
given more, guide than with V and flat, 
however the thrust cannot he centralized 


■unless the V is cut. away or spread, then 
must be more massive, 



Wear in the machine’s ways can be ad¬ 
justed for, prolonging useful life. 

Both the square-type and the dovetail 
gibbed way provide a good guide against 
“shift” in travel. The thrusting element 
can be symmetrically placed. An off-center 
weight is well-guided. 

The dovetail is a practical design to re¬ 
sist heavy lifting or side pressures which 
result from “overhang,” or from large, 
overhanging workpieces, or from heavy 
machining cuts. 

The square-type gib is geometrically 
more correct for producing accuracy. How¬ 
ever, neither gibbed design is capable of the 
ultimate in accuracy. When the gib is ad¬ 
justed for wear, there is not an identical 
repeat of fit or alignment. 

Commonly, a machine’s ways wear in 
the middle of the travel. Adjusting the gib 
for wear can thus only be a compromise 
between a good fit in the center and a good 
fit on the ends of the travel. 

The square-type gibbed way can use a 
master that encompasses both rails; the 
dovetail-type cannot. Neither design is 
truly self-checking by reversal. 

3. The Central V, Fig. 78 

Although less frequently used in machine 


fig. 79 —The double V-way design with 
off-center -weight. 
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design, the central-V would be thought a 
more desirable alternative to the V and 
flat. The thrusting element can be cen¬ 
tralized under the V. An off-center work- 
piece would be more adequately guided. 

Its infrequent use may be explained by 
thefact thatit is often impractical to mount 
the thrusting element directly under the V 
fin which case the central V loses much of 
its value), or because the central V requires 
two outer flats for support. 

As with the V and flat design, it would 
not be practical to use a master which en¬ 
compasses all rails. 

However, the table can be reversed on 
the bed as a self-check. No gib is necessary 
with the central-V design. 

4. The Double V, Fig. 79 
The double-V provides the best guide for 
straightline travel. The thrusting element 
is symmetrical between the V’s. Each rail 
is subject to uniform friction. 

From Fig. 79, it is seen that no matter 
where the workpiece is placed, there is 
always weight on a V to enable it to guide. 

The symmetrical way construction re¬ 
sults in the most uniform wear, and the 
longest wear-life. 

The double-V master, it will be remem- 
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FIG. 80 —When a way-form less than 90° 



Force (holding) 




bered, encompasses both rails and can be 
reversed as a self-check (see page 44). The 
bearing of the machine ways may also be 
self-checked by reversal. No gib is neces¬ 
sary with the double-V design. 

The 90° Form V-way 

Similarly, the advantage of the 90° form is 
best understood by examining the alterna¬ 
tives to it. 

With a form less than 90°, Fig. 80: 

1. There is a weaker cross section; 

2. Difficulty is encountered in accu¬ 
rately scraping to a bearing; 

3. There is a wedging action as the table 
slides on its mate, causing greater 
resistance; 

4. A small amount of wear rapidly di¬ 
minishes accuracy. 

With a form of more than 90°, Fig. 81, there 
is less guide against “side shift.” In fact, it 
would act as an inclined plane. 

It has already been shown that the 90° 
form has the most direct relationship to the 
flat plane. If some angle slightly more or 
less than 90° represents the ideal way-form, 
certainly the 90° form is close enough to 
that ideal to be the best practical choice, 
Fig. 82. 


5. THE MOORE NO. 3 DESIGN 


Based on the analyses already presented, 
it was concluded that the ideal design for a 
coordinate machine of 11 inches x 18 inches 
[279 mm x 457 mm] travel* should incor¬ 
porate the following, Fig. 83: 

1. 90° V-ways; 

2. A compound-type configuration; 

3. Double V construction on all three 
axes. 


*It will be seen that in the larger model Measuring 
Machine, something more akin to the planer-type 
machine was selected. 


fig. 82 . -The 90° form V-way is the 

best guide for straight-line travel and has 
the most direct relationship to the flat plane. 
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FIG. 83 —The Moore No. 3 design 
employs double T 7-ways on all three axes, 
(jig borer shown). 


The base of the machine supports cross¬ 
slide and table in their full travels. Double 
V-ways in the base guide the cross-slide 
and fully resist any side shift. In addition, 
two outer flats prevent deflection of the 
cross-slide member, either from table or 
workpiece weight. 

Double V’s in the cross-slide guide and 
support the table in its full travel. 

From Fig. 84, it is seen that with the 
double V design, even when the table is at 
the extremes of its travel with weight 
added, deflection is prevented and the 
cross-slide is still guided for straight-line 
travel. Thrust of lead screws in both axes 
is centralized between the V’s. 

In contrast, if this same example is 
applied to some form of V and flat design, 
Fig. 85, so often considered the ideal, it is 
evident that one V is not a sufficient guide 
against “side shift.” 

Fig. 86 shows the machine members that 
make up the Moore design—base, cross¬ 
slide, table, column, housing. The hardened 
and ground ways are bolted to accurately 
scraped surfaces and lapped in place to 
final accuracy. Mating cast iron way sur¬ 
faces are scraped to match the lapped ways. 

Two closely related reasons dictated the 
decision to use hardened, ground and 
lapped ways: 1) The attainment of accu¬ 
racy. 2) The prevention of wear. 

1. Attainment of accuracy: 

a. Finer corrections are possible in 
the lapping operation. 

b. Coordinate movements must be 
as friction-free as possible to as¬ 
sure that accuracy of positioning 
is commensurate with the poten¬ 
tial accuracy of the lead screw. 

2. Prevention of wear: 

a. The wear-life of the hardened, 
ground and lapped way construc¬ 
tion is estimated to be at least 10 
to 1 over that of the conventional 
cast-iron to cast-iron way design. 

b. The consequent reduction and 




fig. 84 —The Moore No. 3 design double 
V-way provides a good guide for 
straight-line travel since the weight is 
always over one of the V’s. 


fig. S5 —When the table and weight are 
off-center, the V and flat have lost muck of 
their ability to guide. 
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J; 4 g. 80 — Mupline members that make up 

the No, $ desigti . darting from bo Hat% 

right, dock-wise — borne, cross -slide, table, 
quill housing omul column. 
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almost complete elimination of 
wear insures retention of the ini¬ 
tial accuracy of straightness and 
squareness of the movements. 

SCRAPING MASTERS 

Absolute interchangeability of the ma¬ 
chine members is not feasible when working 
to millionths of an inch [tenths of a pm]. 
There are economies to be realized by 
attaining a degree of interchangeability 
up to the point of final mating. This is 
accomplished through the use of highly 
accurate “scraping masters” of three forms, 
Fig. 87: 1. Flat masters, 2. Female double-V 
masters, and 3. Male double-V masters. 
The origin of these masters and their con¬ 
tribution to accuracy have already been 
described. 

At each operation, such as the scraping 
of the female V’s in the base, two masters 
are provided, one for roughing and one for 
finishing. After approximately eight opera¬ 
tions, the finishing master is worn too 
much to be relied upon and is now used as 
the roughing master. It in turn is replaced 
by a freshly scraped master. The roughing 
master is completely re-scraped. To follow 
such a schedule on a production basis, an 
adequate inventory is kept of all the vari¬ 
ous types of masters needed. 

1. Base, Fig. 88 

The female V’s and the fiat ways in the 
base are not used as way surfaces, but are 
to receive the hardened way inserts. They 
are nevertheless scraped to 0.0001 inch 
[0.0025 mm] accuracy in order to achieve a 
good mounting surface for the hardened 
ways, and to minimize the subsequent 
amount of lapping on the hardened ways. 

The master used on the flat ways is 
sufficiently rigid for the tolerance at this 
stage to transfer its straightness to the 
flats, but not enough to prevent twist. In¬ 
spection for twist is by means of a two- 
footed twist gage. 





fig. 89 — No. 3 base, inspection of 
parallelism of V’s to flats. 




The female V’s in the base are scraped to 
a male double-V master. Inspection for 
vertical straightness and parallelism, as 
well as parallelism to the now-finished flats 
is accomplished by means of a “bridge” 
fixture (see “Geometry,” page 77). The V- 
master is also continually reversed. 

A special table, to which is fixed an in¬ 
dicator, Fig. 89, is used to cross-check 
parallelism of the flats to the V’s. The 
masters are depended upon to produce 
uniform center distance of the V’s. 

2. Cross-Slide, Fig. 90 

The fixture shown in Fig. 91 has hardened, 
ground and lapped V’s of 8.5000 inch 
[215.900 mm] center distance. These V’s 
are parallel to the scraped top surface of 
the fixture and also square to two scraped 
end posts. 

The top V’s of the cross-slides are first 
scraped to a male double-V master so that 
they may be used to rest the slide on the 
fixture without deflection. 

While mounted on the fixture, the follow¬ 
ing geometry, Fig. 92, is scraped into the 
cross-slide: 

1. Vertical straightness of the V’s and 
flats; 

2. Vertical parallelism of the V’s to each 
other and to the flats; 

3. Uniform thickness of top to bottom 
ways; 

4. Squareness of top to bottom V’s. It is 
important to note that the squareness 
of the coordinate axes is embodied in 
only one machine member, and has 
to do with the relationship of top to 
bottom V’s of the cross-slide. 


fig. 90 — No. 3 cross-slide, rough-scraped. 
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FIG. 92 — Squareness of the compound-slide hardened. V's of the fixture on which the 

Vs is established, by traversing a table cross-slide is now mounted. Other features 

holding an indicator past end posts. The end. which may be inspected, are shown by the 

posts h*.'ve been scraped, square to the two other indicators on stands. 
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FIG. S3 -No. 3 table. 




3. TABLE, Fig. 93 

The V’s in the table are scraped to a male 
double-V master. The table-top is in¬ 
spected for parallelism to the V’s while 
resting on the master surface plate, Fig. 
94. Inspection for parallelism of the V’s to 
table top is performed by indicating with 
an indicator and stand across a gage held 
in the V’s (performed by operator, Fig. 94). 
The sides of the table are lastly scraped 
parallel to one another, parallel to the V’s, 
and square to the table top. The indicator 
and stand setup to inspect parallelism of 
sides to V’s is shown at the far end of the 
table, Fig. 94. 

4. COLUMN, Fig. 95 

A double-V master is used to scrape the 
column ways. The surface used to mount 
the column to the base is scraped flat and 
roughly square to its V’s, Fig. 96. Since 
adding any appreciable weight to the col¬ 
umn will alter its relationship to the base, 
final squaring of the column is done after 
all parts are assembled to it. 

5. QUILL HOUSING, Fig. 97 

An abbreviated study of the “operations” 
on the quill housing affords an interesting 
example of the painstaking procedures re¬ 
quired for the attainment of accuracy. Fig. 
99. Operation 12, jig grinding the bush¬ 
ings is shown in Fig. 98. 


FIG. 94 — While the table is resting on 
the surface plate, the parallelism of its 
V's to its top is inspected, as is the 
parallelism of the V’s to its sides using 
an indicator and stand with a knife edge 
(far righ t). 
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-No. 3 Measuring Machine 
qu i-ll-h on sing ran gh-scraped . 


i. 3 Measuring Machine 
coluntH — rough-scraped. 


PIG. no- The column is inverted and its 
mounting surface to the base is scraped 
roughly flat and square to its own V’s. A 
special squaring master is fastened to the 
celumn V’s and used as a datum from 
which to measure. 


FIG. 98 —To establish alignment of the quill 
to the housing ways, the hardened bushings 
in the quill housing are jig ground while 
the quill housing is mounted on a sturdy 
angle iron. 





















FIG. 99— An abbreviated “methods sheet” 
affords an interestin'! example of the 
procedures required for the attainment of 
accuracy. 

Note that there are four separate 
scrapings, plus a lapping operation on 
Ike V’s of the quill housing. 


METHODS 


Date- 

First Load Center— Planer - 0 . S . 


Order No. No. Pcs. This Lot 


1 


Material — G. D. Meehani te 
Size — To Pattern 
Form _ Casting 

NO. | — " _ 


OPERATION 


Part No.— 4870 

Part Name— Quill Housing - 2nd Half 

Std. Lot— Req Per Mach.— 

~ _ ! TOOLS REQUIRED 


Class- #3MM 
#l-l/2MM 


Check casting for mis-match. 

Plane back section & v's to B/P gage. Plane 
to 9" dia. 


Rough scrape V's. Assemble hardened ways (no. 
5064 2 req'd.) 


10 


Season. 


11 


Re-scrape V's and scrape pad. 


Grind bushings to size leaving .0006" - .0007" 
for lapping. 


13 


Lap quill housing bushing to plug gage. After 
rescrape finish lap bushings and do necessary 
bench work to #4870 (Quill Housing) before 
painting. After painting clean up for assembly. 


15 


Remove harden ways (quill housing), check 
bearing with master. Re-scrape using plug in 
bushing hole to check alignment. Final check 
after ways are reinstalled - Master #8552. 


17 


Rescrape column Vee's to new master. Match 
quill hsg. to master. Assemble and lap harden 
ways to match column ways. Scrape for alignment. 
Lap quill hsg. ways with 38-1200 compound to 
column ways. 


Alignment of quill with column upright .OOOO25 

T.I.R. 
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fig. 300 --The ■master flat lav is scraped 
accurately, Uteri impregnated uniformly 
with diamond lapping compound. It is 
used to relate, the two outer hardened flat 
ways. 


FINAL GEOMETRY 

The procedures, techniques and masters 
described here are common to the No. 3 
Jig Borer, No. 3 Jig Grinder and No. 3 Uni¬ 
versal Measuring Machine, which are the 
same in their basic designs. The only essen¬ 
tial difference is that the No. 3 Universal 
Measuring Machine is carried to a higher 
order of accuracy. In regard to geometric 
accuracy, the higher level of accuracy of 
the Universal Measuring Machine takes 
place in “final geometry.” For consistency, 
all operations and tolerances in this section 
will be referring to the Measuring Machine. 

Trueness of geometric relationships— 
straightness, parallelism, center-distance, 
and the fit between matched, sliding sur¬ 
faces—is achieved through use of a com¬ 
bination of precise fixtures and gages, 
autocollimators, electronic indicators, vari¬ 
ous sizes of master laps and checking mas¬ 
ters—all used in temperature-controlled 
environments. 

The Master Laps 

The master laps, both flat (shown in 
Fig. 100) and double V type, can be com¬ 
pared to the scraping masters in that they 
are both scraped to final accuracy by 
identical means (pages 35-45). The master 
laps differ, however, in that they are lastly 
impregnated uniformly with diamond lap¬ 
ping compound. Thus, the scraping master 
imparts a bearing to which the ways are 
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miii<ter for the hardened ways is in the 
foreground. Behind it is the master lap, 
on which is a diamond paste applicator 
and charging block. Bearing range and 
bluing are used to cheek bearings, and 
gloves to minimize the conduction of body 
heat into masters. 




kig. 102 —The master lap is too rigid 
to bend locally, rigid, but may deflect 
as much as 70 millionths of an inch 
[ 0.0018 mm\ overall, left. 
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cut, while the master lap itself cuts the 
ways. In Fig. 101, a scraping master is 
in the foreground and the lapping master 
behind it. 

There is a difference in the type of 
surface which results. The scraping opera¬ 
tion is used to produce a series of uniformly 
high spots in an unhardened material, 
such as cast iron. The master laps are used 
to produce a very high micro-finish in the 
hardened steel ways. 

The master lap, although accurately 
scraped, does not automatically lap out 
the slight errors remaining in the hardened 
ways after being fixed to the machine. It 
will be recalled (see page 26) that elartic de¬ 
flection takes place. Small, local errors are 
eliminated, but not those of overall bow 
and twist, Fig. 102. 

The operation of lapping requires that 
a load be applied so that, in the process, 
the lap wears as well as the machine ways. 
If used too long, the lap rapidly loses its 
accuracy. 

The master laps thus will not correct 
geometry strictly by themselves. For this 
reason, the lapping operation involves all 
four of the following steps: 

1. Use of small hand laps, Fig. 103, 
allows the ways to be lapped locally, 
Fig. 104; 

2. Use of scraping masters, in this ap¬ 
plication more properly termed 
“checking masters,” which aid in the 
hand-lapping by indicating areas of 
bearing, Fig. 105; 

3. Frequent inspection (again the em¬ 
phasis is on quick, accurate, fool¬ 
proof checks); 

4. A more discriminate use of the master 
laps. 

Wear of the master lap is evidenced by 
its failure to agree with the checking 



fig. 104— Hand-lapping of the hardened 
double V requires an experienced hand 
and a thorough understanding of the 
geometry involved. If the surface is lapped 
too deep locally, many additional hours 
may be spent bringing the total surface 
down to this low point. 





master when both are rubbed on the ways 
for a bearing. 

The reader might ask, “Why not cor¬ 
rect the hardened ways first, then fix them 
to the machine?” The answer is that 
bolting causes distortion, Fig. 106. Also, 
the ways do not seat exactly. They must 
be corrected in place to assure correct 
geometry. This is the reason behind the 
master laps. The ways to which the hard¬ 
ened ways are fixed have been scraped 
very accurately in order to provide good 
mounting surfaces and also to minimize 
the amount of lapping necessary. 

THE ELECTRONIC 
INDICATOR AND THE 
AUTOCOLLIMATOR 

Two measuring instruments—the electron¬ 
ic indicator and the photo-electric auto¬ 
collimator—have become virtually “stan¬ 
dard equipment” for the precise operations 
required in final geometry. It is appropri¬ 
ate here to include a brief description of 
these instruments and their use. 

The introduction of the electronic in¬ 
dicator, in particular, has contributed sub¬ 
stantially to a whole new plane of accuracy. 

Yet, the potential level of accuracy in 
which these instruments permit one to 
work is only achieved in fact by creating 
the proper environment, taking precautions 
against the errors which may be introduced 
by the instruments themselves (or through 
their use), and by the development of spe¬ 
cialized supporting equipment. 

The Electronic Indicator 

The mechanical dial indicator has long 
been the workhorse of the workshop. With 
modern designs, Fig. 107, one can be fairly 
certain of readings within 25 millionths of 
an inch [0.0006 mm]. Sometimes, it is nec¬ 
essary to gently tap its mount with some 
small rigid object to overcome any friction, 





FIG. 106- The three variations in surface 
appearance are very shallow depressions 
(local errors) in the flat ways revealed after 
the application of the master flat lap—in 
this case caused by boiling the ways to the 


base. This example illustrates how 
sensitively local errors are revealed by the 
master laps and also why the hardened 
ways must be corrected in place. 



fk;. 107 — The mechanical indicator is a 
very necessary shop measuring tool. 

However, in those applications where an 

indicator is required, but in tolerances to 

millionths of an inch [one or two pm 

or (ess], the electronic indicator is generally 

preferred. gg 









FIG. I os— Schematic diagram of a typical 
electronic indicator 

Court Osy of Federal Products Corporation, Providence. Rhode Island. 




or “lag,” of the internal mechanism when 
making readings. Such a precaution, to¬ 
gether with the relatively high gaging 
force, lends some uncertainty to measure¬ 
ments below the 25 millionths of an inch 
[0.0006 mm] figure.* 

The practical limit reached on magni¬ 
fication by mechanical means was over¬ 
come with the introduction of the electronic 
indicator. This instrument is made up of a 
gaging head and an amplifier. 

The gage head may consist of one of 
several kinds of transducers to which is 
brought a continuous alternating voltage. 

In a typical system, Fig. 108, mechanical 
movement of the contact probe alters the 
output or voltage of the gage head, pro¬ 
portional to displacement. The small alter¬ 
ation in voltage is amplified by means of a 
sensitive meter (except in the case of the 
capacitive-type head), to be read in terms 
of units of length. 

Switching selection enables a convenient 
range of magnifications in one instru¬ 
ment. Magnifications are available of up 
to 100,000 to 1, where 1 millionth of an 
inch [0.000025 mm] is easily discernible. 

A more detailed analysis of the circuitry 
is not included here since this information 
is well covered by suppliers and other 
sources.** 

The electronic indicator has many func¬ 
tional advantages: 

1. High magnification—ranges extend 
up to 100,000 to 1; 

2. A range of magnifications in one in- 


*Mechanical indicators to “millionths” magnifica¬ 
tion do exist, but are more vulnerable to damage, 
harder to read and to use, and not as universal. 
They are more suitable for use in a fixed position, 
being too awkward, for example, in general surface 
gage use. 

**See “Air and Electronic Gaging,” American 
Machinist , October 12, 1964; L. O. Heinold, Jr., 
“Electric and Electronic Gages” in Handbook of In¬ 
dustrial Metrology (ASTME Publication, 1967), pp. 
232-245; Ted Busch, Fundamentals of Dimensional 
Metrology , pp. 225-227. 


fig. ioo —Electronic indicators may suffer 
a loss of linearity ever a period of time, 
so they should be re-calibrated on a regular 
calendar basis. Even with this precaution, 


the best measuring practice is to avoid 
large ranges of meter readings and to use 
the indicator as much as possible as a 
“nulling” device. 
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fig. no —Careless handling detracts from 
the accuracy of the instrument. The operator 
has allowed the cord leading to the gaging 
head to rub against the working area, 
causing erroneous readings. 



strument—which adds greatly to 
convenience in line-up, initial refer¬ 
encing, choosing the right magnifica¬ 
tion for the job, etc.; 

3. Light gaging force—1.5 oz. [43 grams] 
and even less; 

4. Ruggedness—since amplification is 
by electronics rather than by delicate 
mechanical parts; 

5. Immediate response—due to use of 
electronics; 

6. Flexibility—with the proper sup¬ 
porting tools, the electronic indicator 
has an almost unlimited applicability 
to inspection procedures. 

The following may be considered pos¬ 
sible sources of error from the instrument 
itself and may have a bearing on working 
accuracy: 

Repeatability— or a short run ability to 
repeat readings. 

Drift—T he indicator may be relied upon 
for a datum but wanders over a period of 
time. 

Temperature—T he effect of temperature 
change, apart from the temperature of the 
workpiece, may alter the calibration of the 
instrument. 

Magnetism—S ome makes of electronic in¬ 
dicators may be influenced by even the 
slightest residual magnetism. 

Voltage Fluctuation—A n indicator that 
works from ordinary line current may be 
subject to short-run wandering due to volt¬ 
age fluctuation (especially noticeable in the 
higher magnification range of 50,000 and 
100,000 to 1). 

Battery Deterioration—W hile a bat¬ 
tery-operated indicator may be more sen¬ 
sitive and stable over the short run (not 
being subject to voltage fluctuations), it 


may be prone to a more sudden loss of cali¬ 
bration as the cells lose their charge. 

Loss of Linearity— Periodic adjustments 
for linearity are expected, but short-run 
changes stemming from unstable compo¬ 
nents can be dangerous in the inspection 
procedure. 

Degree of Linearity— It should be con¬ 
tinually borne in mind that the electronic 
indicator is a comparator, with its own lin¬ 
earity tolerances; if habitually relied upon 



for wide ranges, the greater the error from 
non-linearity. 

The essential point is that the foregoing 
potential sources of error can impair the 
accuracy of determination unless proper 
precautions are taken. One device for main¬ 
taining the calibration of electronic indica¬ 
tors, Fig. 109, is a hardened, ground and 
lapped wedge, tapered accurately 0.0003 
inch over 7 inches [0.0076 mm over 178 mm] 
or 0.000010 inch [0.00025 mm] per line in¬ 
terval. Another is a voltage stabilizer which 
gives consistent voltage to the 0.000001 
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fig. ill -The photo-electric autocollimalor, 
with a suitable target mirror, such as that 
shown in the foreground, will inspect 
straightness of a machine’s ways in both 
the horizontal and vertical plane. 



High intensity lamp 


Condenser - 

Monochromatic filter - 

Target graticule- 

Optical micrometer refractor block 
Eyepiece graticule \ 

Eyepiece 


Beam splitter 


Micrometer drum 


Beam splitter 
Vibrating slit 


Objective 


Reflector 




Photocell 



Meter 


FIG. 112 -- Photo-electric autocolUmator 
schematic diagram. 

Model TA-5 (monochromatic) is shown. 

Courlcsy of ihy Hank Organization, Hi Igor-Walls Division, London, England. 
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fig. A typical series of autocollimator 
readings is recorded. Actual error may be 
determined algebraically as shown. A. 
graphic representation of actual error is 
easiest to visualize for the purpose of 
corrective lapping. 


inch [0.000025 mm] indicators used in cali¬ 
brating step gages (see page 182). 

There are, in addition, sources of error 
in the environment, such as dirt, oil, air 
film, stability of the setup, finish and geom¬ 
etry of the workpiece, deflection, angle of 
the probe tip to the work, vibration, “tem¬ 
perature,” parallax, etc., not related to the 
instrument itself. Again, these and other 
potential sources of error are well described 
by suppliers. These can only be eliminated 
through the education, experience and 
awareness of the user. A common operator 
error is to allow the cord leading to the 
gage head to rub against nearby surfaces, 
causing invalid readings, Fig. 110. The 
proper method to avoid this is shown in 
Fig. 109. 

Autocollimator 

The auto collimator is an optical instru¬ 
ment used in conjunction with a reflecting 
mirror to accurately measure very slight 
deviations from a datum angle. Briefly, a 
target line, placed at the principal focus of 
an objective lens, is illuminated. The emer¬ 
gent collimated (parallel) rays carrying the 
image are reflected back into the instru¬ 
ment by a suitable target mirror. When this 
mirror is tilted through an angle 6, the re¬ 
flected image rays are tilted through 26 and 
the displacement of the image in the in¬ 
strument is measured by a micrometer- 
microscope in the instrument. By turning 
the dial of the micrometer screw, the image 
is re-zeroed at each reading between two 
parallel setting lines viewed through the 
eye-piece. The angle of tilt can be measured 
in the horizontal or the vertical plane and 
is read directly on the micrometer dial. 

Operator fatigue and possible error in 
setting have in recent years been overcome 
with the development of the photo-electric 
autocollimator. The instrument is shown 
in Fig. Ill and a schematic of its operation 
in Fig. 112. The idea for this system origi¬ 
nated in England with the National Phys- 
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fig. n.4- When “stepping-off” the target 
mirror, the requirement that the steps be 
related must be carefully adhered to (above) 
if the readings obtained are to be valid. 
Failure, to do so (lower) may lead to error 
in calibration. 


Secondsof arc 






FIG. 11/5— Air currents in the optical path 
between the autocollimator and the target 
mirror cause fluctuations in the readings 
obtained. This effect becomes progressively 
more damaging to accuracy as the distance 
from autocollimator to targe! mirror 
increases. 



ical Laboratory. It consists of a vibrating 
slit, a photoelectric detector, and an elec¬ 
tronic amplifier for magnified viewing on a 
meter. 

The auto collimator, with suitable re¬ 
flecting surfaces or optical gages, is thus 
capable of calibrating straightness, flat¬ 
ness, squareness, or the division of the 
circle. 


Checking Straightness with the 
Autocollimator 

Checking straightness with the autocolli¬ 
mator requires a simple setup—a rigid 
mounting surface for the autocollimator, 
and a two-footed mount to which the mir¬ 
ror is fixed. 

Three elements are involved in the deter¬ 
mination of straightness by “stepping- 
off” the mirror: 

1. The angle of each step; 

2. The length of each step; 

3. The relation of the steps to one 
another. 

The autocollimator measures the first ele¬ 
ment, the angle. The second element, the 
length of each step, is established simply 
by using mirror-mount feet with some nom¬ 
inal center-distance length, (if element No. 
2 is absent, the test cannot be made). The 
third element—relating the steps—is ac¬ 
complished by positioning the forward foot 
of the mirror-mount at that point where 
the rear foot previously rested, for the full 
length of the surface to be inspected. 

In this manner, the autocollimator read¬ 
ings are translated into a total calibration 
of the straightness of the bed, Fig. 113. 

Generally, the choice of center-distance 
of the feet is a compromise: if too short, 
the greater number of readings required 
may accumulate errors in calibration; if 
too long, then local errors are not shown. 

If element 3 (inter-relationship of each 
step) is not strictly enforced, the results of 
the inspection are invalid. Using Fig. 114 
as an exaggerated example, if the steps are 


not related, no error is shown, although 
the actual error may be many seconds. 

Many of the sources of error already 
attributed to the electronic indicator apply 
as well to the autocollimator, in addition to 
which should be added, fluctuations from 
air currents, Fig. 115. Errors can be intro¬ 
duced through use of the instrument: the 
flatness and reflectivity of the target mirror 
should be of high quality. 

Where both the autocollimator and the 
target mirror gage can remain fixed, ex¬ 
tremely close readings may be taken and 
repeatability is excellent. Conversely, when 
mirror or instrument must be moved often, 
great care is required. For example, in¬ 
specting a surface plate by autocollimation 
involves an accumulation of readings, a 
time lag, perhaps a temperature change. 
Much depends on how accurately the tar¬ 
get mirror can be moved. Errors are also 
easily introduced when the autocollimator 
itself must be moved to a tie-down point. The 
autocollimator should be used to show over¬ 
all deviations. Where possible, local devia¬ 
tions are more truthfully shown by rubbing 
for a bearing. The fabrication of the 48-inch 
[1219 mm] master surface plates already 
described is a good example of this point. 

While the autocollimator is an indispens¬ 
able instrument, it will be seen that certain 
inspection procedures are faster and more 
directly interpretable by using the elec¬ 
tronic indicator in conjunction with accu¬ 
rate gages. 

Base Flat Ways 

The hardened, flat ways, which have been 
ground in pairs flat and parallel to .000125 
inch [0.003 mm] are bolted to the base on 
the scraped, outer flats. 

Two types of error may be inspected for: 

1. Straightness of each way; 0.3 seconds 
of arc—equivalent to a convexity or 
concavity of 5 to 10 millionths of an 
inch [0.1 to 0.2 pm] over the length 
of the ways. 
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fig. 116— One technique ftr measuring 
parallelism of the flat ways utilizes the 
three-footed twist gage—and simultaneously 
provides another working-check of 
straightness of each flat way. This method 
requires the use of a master surface plate 
as a. datum and is self-checking. 
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FIG. 117 — Use of the two -footed twist gage 
is the most reliable method for deter muting 
pure twist. A self-check only requires that 
the original setting he repeated. The operator 
is settling out the hysteresis effects in the 
twist gage by tapping it with a pencil. 



2. Twist, (parallelism of outer flats over 
their full lengths): 10 millionths of an 
inch [0.00025 mm]. 

Straightness of each flat way is inspected by 
a photo-electric autocollimator. 

Twist is inspected by two methods. In 
the first, the three-footed twist gage uses a 
master surface plate as a reference. To be 
correct, the flat ways must have the same 
reading as the master plate. The master 
plate itself is proved when the same read¬ 
ings are obtained upon turning the twist 
gage 90° on the plate. The three-footed 
twist gage also provides another working- 
check on straightness of the flats, Fig. 116. 

In a second method to check “twist,” the 
two-footed twist gage is used. In this case a 
self-check only demands a repeat of the 
“zero,” Fig. 117. 

Correction of the ways is accomplished by 
a small hand lap and also a master flat lap. 

By way of example, a “twist” in the 
flats is corrected by lapping the high ends, 
tapering gradually to the middle with the 
small hand lap, Fig. 118. This is followed by 
the large lap which blends over small local 
deviations and also relates the two ways. 

The tolerance of straightness and paral¬ 
lelism of the flats is 10 millionths of an inch 
[0.00025 mm]. The flats could be said to 
represent a master surface plate built into 
the base; they become a reference for all 
subsequent measurements of geometry. 

Base V-Ways 

The hardened V-way inserts are next fixed 
to the scraped female V’s in the base. Lap¬ 
ping the V-ways is a similar procedure to 
that employed in correcting the flats. A 
small hand lap and a large female double-V 
master lap are used. 

Potential errors in the V include all those 
encountered in constructing the female 
double-V master, such as lean, center- 
distance, form, twist, straightness, except 
that errors are removed by lapping rather 
than by scraping. 


fig. ns —The method of hand-lapping to 
remove a twist in the flat ways is illustrated 
here. 






fig. 119— The “bridge” fixture relies on the 
now-finished, flat ways as a datum and inspects 
straightness and parallelism of the V-ways. 

Two indicators are used in order that the 
full SI inches [787 mm] of V-way may be 
inspected from the 20 inches [508 mm] of 
flat mays. 
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FIG. 120 - -Once the base jlat ways and 
V-ways are lapped to geometric accuracy, 
the base becomes the “master,” and the 
cross-slide is scraped to match it by means 
of a bearing check. 


Straightness of each V, both in the ver¬ 
tical and horizontal plane, is inspected by 
autocollimation, using a special V-gage 
with mounted mirror. Tolerance of straight¬ 
ness is 0.5 seconds. 

Twist of the V’s is inspected with a 
bridge fixture, Fig. 119, using the plane 
created by the flat ways as a datum. A 
lapped V-gage having a small indicating 
pad is used as the indicating point. By 
fixing two indicators on the bridge, the full 
31 inches [787 mm] of V-way may be inspect¬ 
ed from the 20 inches [508 mm] of flat way. It 
is only necessary that the V’s be parallel 
and not of identical height, since the cross¬ 
slide will be scraped to match the base. 
Tolerance of parallelism of the V’s to one 
another and to the flats is 10 millionths of 
an inch [0.00025 mm]. 

Cross-Slide 

Three steps in the manufacturing process 
insure that the cross-slide now very nearly 
matches the completed base: 

1. The cross-slide has been scraped accu¬ 
rately to a fixture; 

2. The flat ways are ground alike in 
pairs to a thickness that matches a 
particular base assembly; 

3. All double-V masters in use are of 
identical center-distance. 

In final fitting, the base is used as the 
master and the cross-slide is scraped to 
match. Bluing must in this case be used, 
since the hardened and lapped ways will 
not give up a bearing using rouge. 

Matching the cross-slide to the base in¬ 
volves the following procedures: 

1. Check of the bearing, Fig. 120, must 
be uniform; 

2. Check for sideways movement, Fig. 
121, maximum 5 millionths of an inch 
[0.000127 mm]; 

3. Check for “squash,” Fig. 122, maxi¬ 
mum 5 millionths of an inch [0.000127 
mm]. 



fig. 121 — The cross-slide has been scraped 
to match the base. The operator must not 
see more than 5 millionths of an inch 
[0.000127 mm] sideways movement when 
■pushing and pulling the ends of the 
cross-slide. 
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FIG, 122 —In another cheek for the match 
of cross-slide to base, “squash” should not 
exceed 5 millionths of an inch 
[0.000127 mm]. 


Once fully matched to the base, the cross¬ 
slide is perfectly supported for corrective 
scraping on its upper V’s with a fresh 
double-V male master. Most important, 
it is corrected and inspected under the con¬ 
ditions in which it will be used. 

Squareness may have been altered when 
matching the cross-slide to the base, so it 
is re-inspected, using the V-square of Fig. 
123, which is straight and square within 10 
millionths of an inch [0.00025 mm]. Note 
that this square is theoretically not self- 
checked by reversal since a mismatch of 
its base to the V-way may misrepresent 
true squareness.* However, the V-square is 
not used as a final check, but is a con¬ 
venient tool to discover errors of square¬ 
ness when they may be more easily scraped 
out. The effort spent in establishing square¬ 
ness at this stage proves profitable since 
mounting the hardened ways in the upper 
V’s does not usually alter squareness. 

Twist is inspected by comparing ad¬ 
jacent points in the V’s against a fixed 
indicator. This latter inspection relies on 
the good geometry of the baseways, and 
demonstrates again the necessity for the 
close tolerance specified initially on the 
flat ways. 



*A cylindrical-bodied square for seating in the V is 
also used and is self-checking. However, in various 
stages of scraping, the readings are more misleading 
because they only allow for line-contact of the cyl¬ 
inder in the V. 
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fig. 12a—Prior to mounting of the 
hardened ways in the cross-slide, an 
inspection is made for squareness using 
a square with a V base. 


Although the upper scraped female V’s in 
the cross-slide are not functional surfaces, 
the technique of establishing geometry 
requires that they be made very accurately. 



Straightness of each V is inspected by 
autocollimation. Corrective scraping in¬ 
volves a consideration of vertical and hori¬ 
zontal straightness, twist and thickness, 
while maintaining squareness, form and 
horizontal parallelism. 

Hardened Ways Fixed to Cross-Slide 

Once fixed to the cross-slide, Fig. 124, in¬ 
spection of the hardened V-ways is the 
same as previously followed with the 
scraped V’s except that gages must match 
male V’s. Correction is by small hand laps 
and female double V-laps. Inspection for 
bearing requires female double-V check¬ 
ing masters. 

Tolerance of straightness of each V by 
autocollimation is 0.4 seconds as obtained 
in Fig. 113. 

Tolerance of parallelism of each V: 5 
millionths of an inch [0.000127 mm]. In 
Fig. 125 twist is inspected by comparing 
adjacent points of the hardened V-ways. 

FINAL INSPECTION 
Table 

The table ways are scraped to fit the upper 
rails of the cross-slide. The separate checks 
made on all the rails do not guarantee that 
the axes will perform exactly as inspected, 
so the base is now final-inspected for 
straight-line travel and squareness under 
the influence of all the combinations of 
twist, horizontal parallelism, bow, etc., of 
each rail fitted to the table. 

Final Autocollimator Inspection 

A much more accurate determination of 
straightness is possible with the autocol¬ 
limator once the table has been fitted to the 
cross-slide, for the following reasons: 

1. Only the table must now be moved, 
and the mirror-mount need not be 
touched or handled; 

2. The inspector must now only look 
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FIG. 124 —The hardened and ground 
V-ways have now been fastened to the 
upper scraped female V-ways. Each V 
is inspected for straightness using the 
photo-electric autocollimator. Correction 
is by means of small hand laps and 
female double V laps. 


fig. 125 —Vertical parallelism is measured 
by comparing adjacent points of the rails, 
using the cross-slide movement on the base 
as a datum. The necessity of high 


accuracy of the base slideways is here 
apparent; any initial errors would be 
transferred to each subsequent operation, 
such as the one shown. 
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fig. 12 G —This diagram analyzes one 
particular type of error in the ways — first, 
as it would be shown with the 
autocollimator and target mirror (top), and 
second, as it would appear when using the 
straightedge and electronic indicator (bottom). 
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carefully to record any angular devia¬ 
tion over full travel. 

There are several precautions in the use of 
the auto collimator which may be worth 
mentioning at this point: 

1. It is conceivable that the inspector 
may miss or misinterpret the nature 
of the error, Fig. 126; 

2. The autocollimator, while stable at 
short ranges, fluctuates increasingly 
the greater the distance of the re¬ 
flecting mirror from it, as a result of 
air currents; 

3. The autocollimator should be prop¬ 
erly mounted on the machine in such 
a manner that any deflection caused 
by moving the cross-slide or table 
does not deflect the machine member 
to which the autocollimator is mount¬ 
ed. The practice of providing sepa¬ 
rate support for the autocollimator, 
such as mounting on a surface plate, 
is an undesirable alternative, since 
the orientation of that support to the 
machine may unknowingly shift dur¬ 
ing the inspection process. This shift 
can be caused simply by walking in 
the environs of the machine; 

4. If the error in the ways causes the 
table to bend rather than to change 
its angle, part of the error may not be 
shown, Fig. 127. However, the stiff¬ 
ness of the table minimizes the prob¬ 
lem in this case. Where the table is a 
thinner section, the center-distance 
of the feet of the mirror-holding gage 
should be lengthened so as to include 
bending. 

It was felt desirable to develop a com¬ 
pletely different method which would sup¬ 
plement the autocollimator in inspecting 
straight-line travel. 

Many methods were tried and discarded. 
They included: 

A “high-low” check where two indica¬ 
tors were zeroed on two fixed pairs of 
parallel-lapped flats at each end of the 


fig. 127 —When the target-mirror feet are 
too short a span, full error from bending 
of the table does not register. 
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FIG. 128 —Each time the straightedge is 
used, it is specified that it be turned over 
and the inspection repeated. Duplicate 
readings should be obtained for the check 
to be valid. 


gage and at different heights was employed 
but did not show local error. 

A technique where the Step Gage (see 
pages 180-185) was calibrated at the front 
and back of the table and also at different 
heights above the table (very sensitive, 
since errors of straightness are tremen¬ 
dously magnified, but not convenient for 
“production”). 

A precision level measures deviations 
sensitively, but only errors in the vertical 
plane are shown. Also, with this method, 
two levels reading differentially would have 
to be used. One would be mounted on the 
moving element to measure straightness, 
and the other on the machine base, to 
separate out effects which alter the whole 
machine with respect to gravity (caused by 
the nature of the floor support, or deflec¬ 
tion from movement of the machine car¬ 
riage, or walking in the environs of the 
machine). 

The conclusion was reached that the 
straightedge had many fundamental ad¬ 
vantages when used to inspect straightline 
travel: 

1. The inspection is directly interpret¬ 
able to the operator; 

2. The straightedge is fast to use; 

3. The inspection can be self-contained 
on the machine. The indicator can 
usually be placed in a position where 
the movement of the slideway does 
not introduce misleading readings due 
to deflection: 

4. The straightedge is directly related to 
the flat plane; 

5. The straightedge can be self-checked 
by reversal, Fig. 128. 

But it was also recognized that the straight¬ 
edge method itself had limitations which 
had to be overcome: 

1. Necessity for absolute perfection in 
the straightedge. The design of the 
straightedge which fulfills this re¬ 
quirement is shown in Fig. 129. These 
nitralloy straightedges are 26 inches 




FIG. 129 —Nitralloy straightedges used in 
final calibration of straightness are 26 
inches [660 mm] in length. These gages are 
extremely hard, stable, and lapped to a 
final accuracy of 5 millionths of an inch 
[0.000127 mm] as to straightness and 
parallelism of their sides. 
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fig. 130 —When using a single, fixed 
indicator, and a straightedge travelling 
past it, full error of way curvature may 
not be apparent. 


fig. 131 (Center)—A sudden error, 
especially at the extremes of travel, may not 
be apparent if the indicator is left in only 
one fixed position. 
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Error shown at position 2 

indicator position 

1 2 



in length [660 mm] —extremely hard, 
stable and lapped to a final accuracy 
of 5 millionths of an inch [0.000127 
mm] as to straightness and parallel¬ 
ism of their sides; 

2. Distortion of straightedge due to tem¬ 
perature — Precautions must be taken 
to prevent distortion from handling 
by the use of insulating blocks or 
gloves and care in handling. Further¬ 
more, standard inspection procedure 
specifies that at each check, the 
straightedge is reversed, so that any 
temporary distortion would be im¬ 
mediately apparent; 

3. The straightedge cannot be used until 
the table is matched to the cross-slide — 
It was decided that the straightedge 
would be used as a final, double-check 
following inspection by the autocol¬ 
limator. The requirement that the ma¬ 
chine straightness had to pass two 
completely different methods of in¬ 
spection was felt to be an advantage; 

4. When the error in the machine’s ways 
is in the form of an arc of relatively 
large radius, and a single fixed indi¬ 
cator is read against a straightedge 
moving past it, the error may not be 
shown, Fig. ISO. This is even more 
apparent in the case of a sudden error 
at the extremes of travel, Fig. 131. 

The limitation of a single fixed in¬ 
dicator can be overcome by reposi¬ 
tioning the indicator along the length 
of the straightedge and moving the 
slide once- again as in Fig. 132. The 
major objection to the use of a single 
indicator is that it must be physically 
repositioned as shown in order to in¬ 
sure that error is registered. Once 
moved, however, the original datum 
is lost and the error in the travel is 
not related from end-to-end. 

Another procedure, theoretically 
more correct, is to attach the indica¬ 
tor to the machine table and read it 


fig. 132 —When only one indicator is used, 
the indicator probe must be repositioned 
along the straightedge and the slide moved 
once again. 
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fig. 133 —To inspect straightness of travel 
in both planes, a double indicator is used. 
To inspect straightness in the vertical plane, 
the straightedge is mounted on its points of 
least deflection, (.55fL). 


Straightness in the horizontal plane is 
inspected with the straightedge resting on its 
side. In both cases, the straightedge is 
self-checked bij reversal. 


against a straightedge mounted some¬ 
where on the base. If a mount could 
be secured somewhere on the base, 
this would still be awkward because 
of practical difficulties, such as in 
mounting and aligning the straight¬ 
edge, turning the straightedge over 
for a self-check, and in being able to 
differentiate between curvature of the 
ways and “twist” of the ways. 

In order to make the straightedge 
most sensitive to errors of curvature 
and yet remain convenient in use, the 
following procedure is adopted as 
standard: two indicators are used, 
both attached to a common bar, but 
spread apart 9 inches [229 mm] and 
fixed to an adjustable arm. The ad¬ 
justable arm projects from a sturdy 
mount fixed to the rear of the base, 
Fig. 133. 

The “double” indicator does not 
actually discover any error that 
would not be shown by re-positioning 
a single indicator, but does provide 
the following operational advantages: 

a. The error in the ways is related 
over full travel 

b. By using two indicators spread 
apart as,a permanent part of the 
set-up, any error of curvature is at 
all times visible to the operator on 
at least one of the indicators. 

c. It allows the full 26 inches [660 
mm] of straightedge to be used al¬ 
though machine travel in the “Y” 
axis is only 18 inches [457 mm] giv¬ 
ing slightly greater magnification 
of error at the extremes of the 
travel. 

d. The most typical error in the ma¬ 
chine’s slide way is what might be 
called a “uniform arc,” which may 
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fig. 134 —When checking straightness o} 
travel of a machine’s ways with the 
straightedge resting on its table, a “double 
indicator” is much preferred over a single 
indicator. 
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be either concave or convex. Re¬ 
ferring to Fig. 134, showing a ma¬ 
chine with a concave arc, if both 
indicators are set to “zero” in the 
middle of the travel, movement to 
the left will cause a “plus” reading 
on indicator “A” and a minus on 
indicator “B.” Movement to the 
right will cause the opposite, a 
“minus” on indicator “A” and a 
“plus” on indicator “B”—thus 
giving a doubling effect to the er¬ 
ror. The tolerance of straightness 
in both planes of both axes is 20 
millionths of an inch [.000508 mm]. 
The straightedge is mounted first¬ 
ly on edge on its points of least 
deflection to inspect for vertical 
straightness, and secondly rested 
on its side to inspect horizontal 
straightness. 

In final inspection, when using 
the double indicator method, the 
deviation of readings of both planes 
of both axes must not exceed 20 
millionths of an inch [.000508 mm]. 
Each time, the straightedge is re¬ 
versed for a self-check. 


Squareness 

For final inspection of squareness of the 
X-Y axes, a T-square accurate to 10 mil¬ 
lionths of an inch [.000254 mm] is used, 
Fig. 135. The base rail of the square is 
aligned with the X axis using an indica¬ 
tor. Leaving the square in the same posi¬ 
tion, the indicator is swung to indicate the 
blade by sliding the Y axis. 

Both the square and the machine are 
simultaneously checked by comparing the 
readings obtained when the square is re¬ 
versed, Fig. 136. Shop tolerance of square¬ 
ness is 20 millionths of an inch [0.5 pm]. 

Table Top 

Using a flat master for bearing, the table- 
top is lastly scraped for “thickness” (paral¬ 
lelism to the ways) and for flatness. The 
inspection procedure consists of positioning 
nine points of the table under a fixed in¬ 
dicator by moving cross-slide and table. 
The tolerance specification of 20 millionths 
of an inch [0.5 pm] TIR by this method 
is a severe one, for it necessitates near-per¬ 
fect geometry of the whole base construc¬ 
tion, Fig. 137. 


Column 

The column is scraped to a male double-V 
master and auto collimated for straight¬ 
ness of the ways. 

The column is squared to the base, Fig. 
138, after final assembly to the actual 
conditions under which the machine will 
operate—with housing and all components 
in place. 

The close tolerance specified above on 
the table top now assures a reliable square¬ 
ness check. The cylinder square shown is 
self-checked by reversal. 

Tolerance on squareness is 50 millionths 
of an inch [1.3 pm]. 
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fig. 135 —Final inspection of squareness 
employs a nitralloy T-square, accurate to 
10 millionths of an inch [0.00025 mm]. The 
square is self-checked by reversal. Tolerance 
of squareness of the machine axes is 20 
millionths of mn inch [0.0005 mm}. 
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To find out-of-squareness of machine: 

(I) Machine + square = — 50 

(II) Machine — square = — lOt 

2 x Machine =— 60(0 + (II) 

Machine = —30 

( — 30)+ square = — 50 (sub in I) 
square = — 20 

fActual reading was + 10 but the sign changes due to 
indicating from opposite side 

Readings are in millionths of an inch 


FIG. 136 —Actual squareness error of both 
the machine axes and the T-square is 
determined upon reversal of the T-square. 
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fig. \37—“Thickness” of the machine 
table and flatness of its top are lastly 
established by positioning nine points of 
the table under a fixed indicator. This is 
done by moving the machine axes. 


A tolerance of 20 millionths of an inch 
[0.0005 mm] by this method is severe, since 
it requires near-perfect geometry of the whole 
base construction. 
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FIG. 13S —The column is squared to the 
■plane of travel of the base by using a 
cylinder-square master. The close tolerance 
previously specified on the table top allows 
this to be done with great accuracy. The 
cylinder square is self-checked by reversal. 


























fig. 139— Alignment of the quill to the 
housing travel is inspected in two directions 
by using a lapped round hole. 

The hole is picked up with the housing 
lowered and quill retracted {left). Error in 


both directions registers when the housing is 
raised and the quill extended {right). The 
doubling effect makes this inspection highly 
sensitive. 


Alignment 

Alignment of the quill travel is also lastly 
inspected in its final, vertical position, Fig. 
139. 

Readings taken when revolving an in¬ 
dicator fixed to the spindle, first with the 
housing lowered and quill retracted (left), 
and next with the housing raised and quill 
extended (right), show quill misalignment 
in both directions. The doubling effect by 
using a hole makes this method very sen¬ 
sitive. The holes used are lapped round 
within 5 millionths of an inch [0.000127 
mm] and of high micro-finish. 

Tolerance of alignment is 30 millionths 
of an inch [0.00076 mm]. 
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STANDARDS 
OF LENGTH 




The responsibility for the establishment 
and maintenance of the ultimate stan¬ 
dard of length, which is at present a 
specified number of wavelengths of the 
krypton 86 isotope, rests with the inter¬ 
national and national bureaus of stan¬ 
dards. The accuracy of derived length- 
standards such as gage blocks, precision 
scales, Step Gages, lead screws, and laser 
interferometers depends in large mea¬ 
sure on how truthfully they represent the 
ultimate length standard. 


FIG. 140 —The master lead screw is the 
measuring element in all Moore machines. 
Its accuracy is the culmination of efforts 
in establishing accurate and “traceable” 
standards of length. 





THE STANDARDS BUREAUS 

Modern manufacturing technology is based on precise, reliable dimensional mea¬ 
surements. Ultimately, all of these measurements are comparisons with standards 
developed and maintained by bureaus of standards throughout the world. 

Most bureaus are research centers as well, where the science of precision mea¬ 
surement is continuously refined. The bureaus direct much of their research toward 
the solution of industry’s measurement problems and serve as forums and clearing¬ 
houses for information on measurement. Mass production, automation, advanced 
products—these are some of the benefits directly conferred on mankind by the 
work of the bureaus. The indispensable relationship of the bureaus to the total 
scheme of technology will be continually emphasized throughout this section; for 
it is particularly in the matter of length standards that the need for national and 
international agreement becomes most apparent. By way of emphasizing their 
vital role, on the following pages appear unique contributions by an official of 
each of four leading bureaus, describing the scope of their work.* 


*A good source of additional information: Arnold W. Young, The Bases of Measurement Calibration Facilities in 
National Foreign Laboratories, Paper presented at AO A Standards & Metrology Meeting, Philadelphia, April, 1970. 






BUREAU INTERNATIONAL 
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POIDS & MESURES 


PAVILLON DE BRETEUIL 
F92- SEVRES FRANCE 

TELEGRAMME: POIDMESURE, SEVRES 
TELEPHONE: 037 00-51 


Functions, Activities and Goals of 
the Bureau International 
des Poids et Mesures 

The International Bureau of Weights and Measures was founded 
on May 20, 1875, by the Conference of the Metre in which seventeen 
different countries were participating. At the present time forty coun¬ 
tries are members of the Conference and take an active part in the manage¬ 
ment of the International Bureau, the facilities of which they are joint 
owners. 

The International Bureau is a scientific center and metrological lab¬ 
oratory in its own right, with the main purpose of creating a unified world¬ 
wide system of physical measures. 

Its duties as such are: 

-to establish basic standards as well as reference systems for the 
principal physical measurements and to preserve national and inter¬ 
national masters; 

-to ensure international coordination between measurements of cor¬ 
responding purposes; 

-to determine physical constants and to coordinate worldwide research 
in the field of basic metrology. 

In the years following its creation, the International Bureau, being 
mainly concerned with lengths and masses, concentrated on the prep¬ 
aration of the standards which were distributed to the participating 
governments. Later on, the work on the gas thermometer led to the de¬ 
termination of the standard hydrogen thermometric scale, and the in¬ 
vestigation into the dilatation of nickel alloyed steel enabled C. E. 
GUILLAUME to discover the invar alloy. The accurate determination 
of the mass-volume-relationship of water, the dilatation factors of the 
standard platinum-iridium alloy and the use of invar wires in connec¬ 
tion with geodesic length measurements go back to this same period. 

As early as the nineties, the International Bureau found a new field 
of activities with the determination by MICHELSON and BENOIT 
and later on by BENOIT, FABRY and PEROT of the red spectrum 
line of cadmium. 

Even though it entered the new fields of electrical (1927) and pho¬ 
toelectrical (1933) standards and of ionizating radiations (1960), the 
Bureau did not fail to keep developing its initial activities. 

On October 14, 1960, the Eleventh General Conference on Weights 
and Measures meeting in Paris availed itself of the work done as early 
as 1954 to redefine the meter in terms of the wavelength of the orange 
red spectrum line of the isotope of krypton having a mass number of 86. 

Interferometry is successfully used for measuring material length 
standards and for the determination of gravitational acceleration, with 
unprecedented accuracy. 

Even though the orange red spectrum of krypton is still the primary 
length etalon, laser radiations are now being actively investigated. 

Progress in the fields of electronics and quantum physics seem to open 
up new possibilities towards ever more accurate metrological standards. 
To live up to its vocation, the Bureau shall leave no avenue of progress 
unexplored. 

P. Carre 

For the Director J. Terrien 
10 March 1969 


NATIONAL PHYSICAL LABORATORY 
TEDDINGTON, 

MIDDLESEX. 

The work of the National Physical Laboratory, 

Teddington, England 

The main objectives of the National Physical Laboratory are sum¬ 
marised in the following quotation from the formal opening of the Lab¬ 
oratory in 1902: “To bring scientific knowledge to bear practically on 
our everyday industrial and commercial life, to break down the barriers 
between theory and practice, to effect a union between science and com¬ 
merce.” 

To attain these objectives, a large part of the work of the Labora¬ 
tory is concerned with the establishment and maintenance of basic stan¬ 
dards of measurement and the furtherance of their use by industry. An 
important contribution to this task is made by research carried out to 
develop new and improved standards and better and more accurate 
methods of calibrating them. Intercomparisons are made between the NPL 
standards and the national standards of other countries and those main¬ 
tained by the International Bureau of Weights and Measures at Sevres. 
Industrial reference standards in certain fields of measurement are cali¬ 
brated at the Laboratory in terms of the NPL standards and by this means 
industry is enabled to establish a uniform basis of measurement throughout 
the country in conformity with similar measurements made abroad. The 
latter objective has been facilitated by the establishment of the British 
Calibration Service (BCS) of the Ministry of Technology, and all the 
laboratories approved by BCS in the field of engineering metrology have 
reference standards traceable to NPL. 

In addition to the work on basic standards, the NPL also assists industry 
in the United Kingdom by advising on problems of measurement and 
where appropriate by developing suitable measuring instruments to solve 
these problems. 

L. W. Nickols 
Head, Metrology Centre 
1 May 1969 
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Physilcalisch-Technische 

Bundesanstalt 

Abteilung I 


Braunschweig, 

Bundesallee 100 

Femtpreeher: 5921, Durchwahl Ober 592 
Ferrttchreiber: 9-52822 (plb bSwg) 
Telegromme: Bundetphytik Brauntchweig 
Frochlgut: Brauntchweig—lehndorf 

Exprettgul: 8rauntchweig—Hbf 


Dimensional Metrology at PTB 

The PTB (Physico-Technical Institute of the Federal Republic of 
Germany), formerly named PTR (Physico-Technical Institute of the 
German Reich) was founded in 1887 and hence is the oldest national 
physical laboratory of the world. Its original objectives were: 

1 ) to carry out research in all fields of physics and engineering, thus 
assuring maximum application of the physical and engineering sciences to 
the advancement of industry; 

2 ) to be in charge of testing materials, mechanisms, structures and 
measuring instruments or tools. 

Today, one of its chief concerns is the maintenance of our measuring 
system, particularly the maintenance of the fundamental physical and 
technical standards as well as the invention or development of new 
methods of definition and their control. This, of course, has to be done 
in close collaboration with the other national laboratories. 

In 1969, nearly 1300 people, among the approx. 300 scientists, were 
working at PTB, which is subdivided into the six classical divisions: 
mechanics, electricity, heat, optics, acoustics and atomic physics. 

Dimensional metrology is the prevailing objective of the mechanics 
division which includes, besides others, three length measurement lab¬ 
oratories: length unit, end measures and line measures. 

The length unit laboratory played a leading role at the recent re¬ 
definition of the meter in terms of the wavelength of the orange red 
krypton 86 line. The krypton 86 lamp and useful instruments enabling 
end gages to be measured with light waves, for instance the well-known 
Koesters interference comparator for the measurement of end gages up to 
100 mm (4 in.) and the big Koesters end gage interferometer for the 
measurement of longer gages up to 1000 mm (40 in.) have been designed 
and important research on the application of light waves to length measure¬ 
ment has been carried out in this laboratory. Using the Koesters end gage, 
interferometer gages are measured in terms of a light wavelength in 
vacuo, thus eliminating the uncertainty of the refraction of air. 

The precision is the highest attainable at the present time and is 
limited mainly by the imperfections of today’s end gages and by the diffi¬ 
culty in measuring the temperature more accurately than to 10 -3 degrees. 
Actually the uncertainty of determining end gages up to 1000 mm is of the 
order of 10‘ 8 m. Aiming at the creation of an eventual future length stan¬ 
dard in terms of the wavelength of a laser radiation research on the ap¬ 
plication of lasers to length metrology was started years ago. A xenon-ion 
laser as a rather promising light source for establishing a more suitable 
wavelength standard has been developed. 

Shorter end gages up to 100 mm (4 in.) are calibrated in the end gage 
laboratory using the Koesters interference comparator. The lowest 
attainable uncertainty is ± 0.02//^m (approx. =fc 1/jum). Longer end mea¬ 
sures are measured in this laboratory either by interferential comparison 
or by the use of various mechanical measuring equipment. 
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In the line measures laboratory a 1 m-comparator with photoelec¬ 
tric microscopes is used for comparing line meters by a displacement 
method. Line scales in general can be calibrated either in relation to 
end gages using a Koesters type interferometer or fundamentally using 
a helium-neon laser and a fringe counting technique. Measuring tapes and 
wires up to 50 m are measured in the same laboratory using a tape bench 
or a geodetic tape comparator furnished by the SIP Company. 

In addition to the aforementioned basic length measurement laboratories, 
another group of length metrology laboratories, concerned particu¬ 
larly with industrial measuring problems, is working at PTB. There 
are enough facilities for the measurement of all kinds of gap gages, balls, 
cylindrical plug or ring gages and tapered plugs or rings. An interferential 
method and new types of feelers have been developed for these measure¬ 
ments. The characteristics of thread gages, particularly of the various forms 
of tapered threads according to the requirements of the API-standards, 
also of gears or worm gears can be determined by means of equipment 
designed for the greatest part at PTB. A high precision method is used for 
the calibration of rotary tables, angular index tables, angle dividers, circle 
graduations in general and for checking autocollimators. Conventional 
equipment is available for investigations on surfaces. 

Prof. Dr. E. Engelhard 
Leitender Direktor, PTB 
August 9, 1969 
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U.S. DEPARTMENT OF COMMERCE 
National Bureau of Standards 

Washington, D.C. 20234 


Functions, Activities, and Goals 
of the 

National Bureau of Standards 
Washington, D.C. 20234 

The National Bureau of Standards is organized into the Institute 
for Basic Standards, the Institute for Materials Research, the Insti¬ 
tute for Applied Technology, the Center for Radiation Research and 
the Center for Computer Science and Technology. 

The Institute for Basic Standards provides the central basis within 
the United States for a complete and consistent system of physical measure¬ 
ment, coordinates that system with the measurement system of other 
nations, and furnishes essential services, including measurement and 
dissemination of fundamental properties of matter, leading to accurate and 
uniform physical measurements throughout the Nation’s scientific, in¬ 
dustrial, and commercial communities. 

This Institute serves classical subject matter areas such as: Applied 
Mathematics, Electricity, Metrology, Mechanics, Heat, Atomic and 
Molecular Physics, Radio Standards Physics, and Engineering, Time and 
Frequency, Astrophysics, and Cryogenics. 

The Institute for Material Research assists and stimulates industry 
through research to improve understanding of the basic properties of 
materials, develop data on the bulk properties of materials, and devise 
measurement techniques for determining these properties. 

This Institute’s efforts are in technical fields such as Standard Ref¬ 
erence Materials, Analytical Chemistry, Polymers, Metallurgy, In¬ 
organic Materials and Physical Chemistry. 

The Institute of Applied Technology develops criteria for the evalu¬ 
ation of the performance of technological products and services, pro¬ 
vides specialized information services to meet the needs of the Nation’s 
industrial community, and provides a variety of specialized technical 
services for other Federal Agencies. 

This Institute deals with Engineering Standards, Weights and Measures, 
Invention and Innovation, Vehicle Systems Research, Product Evalua¬ 
tion, Building Research, Electronic Technology, Technical Analysis, and 
Measurement Engineering. 

The Center for Radiation Research conducts programs important 
to the Nation in basic standards, materials research, and applied tech¬ 
nology utilizing radiation and nuclear scientific techniques. 

The Center for Computer Science and Technology provides technical 
services to Government agencies and conducts research in the field of 
automatic data processing, computer language, and systems design. 

The Office for Information Programs promotes optimum dissemina¬ 
tion and accessibility of scientific information generated within NBS 
and other agencies of the Federal government and promotes the de¬ 
velopment of the National Standard Reference Data System and a 
system of information analysis centers dealing with the broader aspects 
of the National Measurement System. 
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The National Bureau of Standards is continually working on means 
to define standards of length, mass, time and temperature in terms of 
natural constants which can be reproduced anywhere with high fidel¬ 
ity by anyone with access to specified technical facilities. NBS coop¬ 
erates with the National Standards Laboratories of many nations and 
the International Bureau of Weights and Measures through committee 
activities. 

A. G. Strang 

Chief Engineering Metrology Section 
Metrology Division NBS 
August 4, 1969 
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fig. 141 —Standards of length and weight 
became necessary as civilizations emerged. 
The oldest of these prehistoric Egyptian 
weight-measures dates to 7000 b.c. 

Courtesy of Science Museum, London. 

British Crown Copyright. 
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1. History of Official 
Length Standards 

Most measurements in the broad realm of 
dimensional metrology are self-proving or 
“natural”; they require no reference to 
outside authority. By way of example, a 
block of granite can be made flat, or the 
divisions of a sundial placed as accurately 
as tools, skills, and the ability to measure 
allow. While measuring devices may be¬ 
come more sophisticated, principles of 
measurement remain the same. 

With the measurement of length, how¬ 
ever, such is not the case. Some authority 
must determine what constitutes an inch 
or a millimeter. Since the length of an ob¬ 
ject varies with its temperature, the same 
authority must further decide at what tem¬ 
perature it is to be measured. A standard of 
length, in other words, is arbitrarily derived. 

The origin of the present standards of 
length and weight dates back thousands of 
years, Fig. 141. It was not until 1960, when 
light waves were chosen as the fundamen¬ 
tal standard, that the unit of length was 
finally related to some unchanging phe¬ 
nomenon in nature. This standard of mea¬ 
surement realizes a long sought-after goal 
of metrologists—the agreement on an 
immutable standard. 

To understand the complexity of the 
length-measuring problem, it is valuable to 
re-trace how standards evolved. 

IN THE BEGINNING 

In his earliest attempts to relate the sizes 
of various objects about him, man com¬ 
pared them to the limbs of his body. Such 
a comparison was logical because these 
“units” could be easily understood by a 
fellow-human. Moreover, they provided 
an adequate and readily available means of 
measurement. 

Not until civilization progressed in tech¬ 
nology and men became more interdepen¬ 
dent did it become apparent that such 


fig. 142 — Moltenjodaro stone weights and 
fragments which have been preserved are 
several thousand years old. The Indus 
civilization, while highly developed, was 


completely destroyed and apparently had 
little influence on subsequent cultures. 
Courtesy of Science Museum, London. 

British Crown Copyright. 
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FIG. M 3—Sacred importance was attached 
to standards of length and weight. Shown is 
a decapitated sculpture of Gndea, Governor 
of Lagash, circa 2175 R.c, On his lap is 
a tablet with the graduated rule. 

Courtesy tithe Louvre, Cliche des Musses Nationaux. 


“units” were inadequate. In their place, 
physical duplicates were made which were 
multiples and sub-multiples of these units. 
These standards of measurement were 
often stored in a temple or other safe place, 
where official copies originated. 

Early cultures felt the need for standard¬ 
ization of measurements of length, weight, 
and volume. Although measuring systems 
most often developed independently, there 
were also many cultural cross-influences. 
A striking example is the agreement be¬ 
tween the ancient Hebraic cubit, 17.60 
inches [447 mm] and the Egyptian short 
cubit, 17.64 inches [448.05 mm]. 

Sarton observes in A History of Science: 
“The outstanding cultural patterns coa¬ 
lesced in the valleys of great rivers in 
northern subtropical regions .... Those 
rivers are the Nile, the Euphrates and the 
Tigris, the Indus and the Ganges, the 
Hwang Ho and the Yangtze, and perhaps 
also the Menam and the Mekong.”* 

Actually, the ancient Chinese and Indus 
cultures seem to have had little influence 
on our own. 

Professor Harkness lists the Babylonian 
or Chaldean system of weights and mea¬ 
sures as the most ancient. He believes it is 
from this culture that the Egyptians de¬ 
rived their system of weights and mea¬ 
sures.** Perry states that the earliest 
Egyptian weight was taken from the prehis¬ 
toric Indus civilization.*** Early Indus 
stone weights are shown in Fig. 142. 

It is impossible to trace with any degree 
of certainty the origin of standards of 
weights and measures, or for that matter 
the exact beginnings of civilizations. It is 
certain, though, that each civilization at an 
early date developed systems of weights 
and measures to which almost sacred im¬ 
portance was attached, Fig. 143. 

Historians may disagree as to which is 
the earliest, but it appears Egypt passed 
on the strongest heritage to western 
culture. 



*George A. Sarton, A History of Science, p. 19. 

**William Harkness, “The Progress of Science 
as Exemplified in the Art of Weighing and Mea¬ 
suring.” Smithsonian Institution Report, July 1888, 

p. 616. 

***John Perry, The Story of Standards, p. 24. 
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THE EGYPTIAN HERITAGE 


Egypt was most favored geographically in 
that it had a long, navigable waterway— 
the Nile. This great river’s periodic over¬ 
flowing fertilized the Nile Valley enabling 
the establishment in Egypt of a stable 
agrarian-based culture. The shifting bed of 
the Nile stimulated advanced technology 
in land survey, enabling rulers to better 
administer and tax their domain. This 
advancement, in turn, led to measuring 
and counting, which together form the key¬ 
stone of engineering, Fig. 144. 

The high level of Egyptian accomplish¬ 
ment was made possible by a continuous 
culture of several thousand years. Un¬ 
fortunately most of Egypt’s treasures 
and monuments were lost, having been 
plundered by a succession of conquerors— 
the Greeks, Romans, Arabs, French, 
British, and the ancient Egyptians them¬ 
selves. Enough remains, however, to make 
the assessment that it was a highly de¬ 
veloped civilization. 

Of the remaining few, the most com¬ 
manding monument to Egyptian ingenuity 
is the Great Pyramid built for Khufu 
(Cheops) of the fourth dynasty. It mea¬ 
sured about 775 feet [236.22 meters] on 
one side alone, and stood nearly 480 feet 
[146.30 meters] high. The mean error in 
the length of its sides was only 0.6 inch 
[15 mm]; in its angle it fell short of being a 
perfect square by only 12 seconds, Fig. 145. 

Egyptian architectural prowess is 
equally evidenced by the obelisk. Though 
many of the great granite obelisks were 
transported to other countries of the 
world, one at Aswan remains because it 
developed a fissure and was never fully 
quarried. If extraction and removal were 
possible, it would measure 137 feet [41.75 
meters] in height and weigh 1,168 short tons 
[1059.6 metric tons). The harnessing of up¬ 
wards of 30,000 men at a time to fashion 
and erect such monuments was surely one 








vie. 14G . The cubit was an important unit 

oj measure in the ancient world. Official 
physical representatives of the cubit were 
made and stored in places of safe-keeping. 
The Jive Egyptian cubits shown are of 
difficrent mater ials. 

Courtesy la Rivista (RIV), May, 1961. 


of immense logistics, requiring organiza¬ 
tion, engineering and well-established 
standards of measurement. 

Bothmer* emphasizes the influence of 
Egypt on Greece and Rome. He states that 
much as “imported” goods are highly 
prized today, so too whatever came from 
Egypt was especially valued. The Egyp¬ 
tians had an undeniable sense of the order 
of things, a sense assimilated and in turn 
disseminated by the Greeks. The Romans, 
in turn, acquired this faculty and passed it 
on to the civilized world. The lasting in¬ 
fluence of Egyptian thinking is very ap¬ 
parent in the matter of standards. To cite 
an example, the length of the ancient 
Egyptian foot, 11.78 inches [299.2 mm] 
does not differ appreciably from that of 
the present foot [304.8 mm]. 

Authorities disagree as to which unit of 
length was the most commonly used. 
Assuredly, the “cubit” (tip of finger to 
bent elbow) was a basic unit of measure¬ 
ment. Others maintain that the fathom 
(length of outstretched arms) was more 
important. Recent comparisons of 5 
Egyptian cubits, using modern techniques, 
lead Professor Scamuzzi of Turin to con¬ 
clude that the most basic unit was that of 
the digit (much as the inch is to the English 
system and the millimeter is to the metric). 
He feels that all other units of measure¬ 
ment, such as the cubit, were convenient 
multiples of the digit.** 

The more ancient Short Cubit was com¬ 
posed of 24 digits, approximately .736 inch 
[18.7 mm] each. The number 24 was ap¬ 
parently chosen because it could be con¬ 
veniently divided. Later, the Royal Cubit 
came into use, probably by adding 4 
digits. The Royal Cubit, composed of 28 

*Author’s conversation with Bernard Bothmer, 
Egyptologist, Brooklyn Museum, Brooklyn, New 
York, May 28, 1968. 

**Ernesto Scamuzzi, “Historical Comments about 
Some Cubits Preserved in the Egyptian Museum of 
Turin,” Technical Journal of RIV (Turin), May, 
1961, p. 20. 



digits, was no longer mathematically con¬ 
venient. Later dynasties wished to pre¬ 
serve its length, however, so that the Royal 
Cubit was restored to its 24 digit sub¬ 
division, making a new long digit of .858 
inch [21.8 mm]. 

Of the five cubits in the Turin Museum, 
Fig. 146, two are of wood (their lengths, it 
should be noted, are suspect since they un¬ 
doubtedly underwent considerable shrink¬ 
age) ; one is wood-core and gold-covered, and 
another is of basalt. Still another is bronze. 

Most interesting of the 5 cubits is speci¬ 
men No. 3. Inscribed on this cubit is the 
digit, its multiples and sub-multiples, and 
both the Short and Royal Cubit, showing 
the transition. A close-up of this specimen 
is shown in Fig. 147. 

BABYLONIAN CONTRIBUTION 

The Babylonians, successors to the Egyp¬ 
tians, were primarily mathematicians and 
men of reflection. Contrary to popular 



pig. 147 — A dose-up of an inscribed cubit. 
The space between each line represents a 
“digit." The digit was a convenient unit- of 
length, much as the ’millimeter is in the 
metric system and the inch is in the 
English system . 

Courtesy La Rivista (RIV), May, 1951. 
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fig. 1.4S —Murks for weights and measures 
of various civilizations. 

Courtesy of Science Museum, London. 
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opinion, they were not such, dedicated ob¬ 
servers and recorders of natural phenom¬ 
enon. Consequently, they were less adept 
at the art of astronomy than is often sup¬ 
posed. Their prime contribution was the 
addition of numerical foundations to the 
existing store of knowledge. Attributed to 
them is the sexagesimal system of num¬ 
bers which has survived to this day. The 
system is still used for navigation and the 
recording of time. 

The number 6 and its multiples were also 
found to have special mathematical rela¬ 
tionships, especially with regard to angles. 
Our present system (see “Dividing the 
Circle,” page 205) is largely of Babylonian 
derivation. 

THE ANCIENTS 
AND PROPORTION 

The Greeks, to whom the culture of Egypt 
next passed, had an acute sense of beauty 
and proportion. They attained a level of 
symmetrical sophistication unparalleled in 
history—best expressed in their statuary 
and architecture. 

The Romans, who rose to world promi¬ 
nence after the Greeks, embraced and ab¬ 
sorbed Greek art, architecture and sense of 
proportion, Figs. 148 & 149. 

The ancients, particularly the Greeks, 
were keenly aware of the beauty, symme¬ 
try, and proportion of the human body. 
They were equally aware of the body’s 
various numerical proportions. 

At first, the numbers 5 and 10 which 
relate to the fingers, were regarded as 
ideal. Later it was felt that 6 and 10 had 
the same attributes of perfection. We see 
the inter-relationship of these numbers and 
their combination 16 to various parts of 
the body. For example, the foot is the 6th 
part of a man’s height; the cubit consists 
of 6 palms and 24 fingers; the foot has 16 
fingers, etc. 

It is not surprising, therefore, that the 
ancients considered these proportions to be 


appropriate for purposes of design. Indeed, 
they can be noted today in surviving Greek 
and Roman art and architecture. 

Vitruvius (circa 77 A.D.), a Roman, wrote 
on the topic of designing temples, Fig. 150: 

“For if a man lies on his back with hands 
and feet outspread, and the center of the 
circle is placed on his navel, his fingers and 
toes will be touched by the circumference. 

. . . Moreover, they [the ancients] collected 
from the members of the human body the 
proportionate dimensions whichappear nec¬ 
essary in all building operations, the finger 
or inch, the palm, the foot, the cubit.”* 

This does not mean that measurements 
were drawn directly from the human body; 
body dimensions were used only as models. 
As might be expected, the actual lengths of 
the units went through many mutations 
and modifications throughout the centuries 
— as described with the Egyptian cubit. 

ROMAN EMPIRE UNIFIED 
STANDARDS 

Along with the expansion of the Roman 
Empire went its culture, and, as might be 
expected, its disciplines governing mea¬ 
surement. 

The Romans divided the Greek Olympic 
Cubit into 12 thumb-widths which were 
called “unciae” or inches. 

As conquerors, the Romans were con¬ 
cerned with the longer distances and estab¬ 
lished the “mille”, or 1000 double steps, 
as a unit of measurement. Since Britain 
was ruled by Rome for nearly 400 years, 
Roman influence pervaded for some time. 

Our present numerical system, a vast 
improvement over the Roman system of 
numbers, appeared at a later date as a con¬ 
tribution of the Arab culture. The system 
was probably derived from the Hindus. 

*Items 17, 23-24 of Vitruvius’ “De Architectura” 
(see Technological Studies in Ancient Metrology by 
Eivind Lorenzen, p. 23). 

It is interesting to note that Vitruvius was highly 
influenced by Hellenic architecture. His technique 
was studied by Michelangelo, Da Vinci and others. 
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fig. 149 — Romano-Greek marble tablet is 
inscribed with the Olympic foot measure 
(, 300- 500 A.D.). The Romans absorbed much 
of the Greek culture. Homan influence 
spread- throughout the entire civilized world. 

Courtesy of Science Museum, London, 

British Crown Copyright. 


Although the cubit and other early units 
of measurement have long since fallen into 
disuse, such units as the “foot” nearly as 
early in origin, and the “inch” have re¬ 
mained in use. They are still the standard 
units of length in English-speaking coun¬ 
tries. 

The decline of the Roman Empire ended 
the monolithic unity it had given the 
civilized world. The warring kingdoms that 
followed only fed on the remnants of the 
Roman order. Hardly any progress in the 
standardization of units can be noted. Up 
to the beginning of the sixteenth century, 
standards seem to have been forgotten. 
Measurements frequently reverted to the 
human body. Only France and England 
had what could be called official standards. 
Germany had no true standard until its 
adoption of the metric system in 1870. 

The definition of a particular unit of 
length was frequently left to the whim of 
ruling monarchs. Some of the more comical 
examples are: A “yard” equals the distance 
from the end of the King’s nose to the tip 
of his outstretched hand;* three grains of 
barley, dry and round, make an inch;** a 
“rod” is equal to the combined left feet 
of the first sixteen men out of church on a 
particular Sunday morning.*** 

EARLY STANDARDS 
IN FRANCE 

The first evidence of a French national 
system of measurements is found during 
the reign of Charlemagne (768-814). Re¬ 
productions were made of the Pied de Roi, 
or royal foot, and divided into 12 inches 
( pouce ) after the Roman system. The 
earliest record of a standard of length is 
the Toise du Grand Chatelet of 1668, said 
to represent one-half the distance (12 feet) 
[3.657 meters] between the walls of the 
inner gate of the Louvre. The genealogy of 

*Definition attributed to King Henry I (1068- 
1135). 

**Definition attributed to Edward I (1305). 

***Germany, 16 th Century. 
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fig. 150 —The human body was used as a 
model by the ancients for measuring and 
proportion. Numerical proportions found in 
the human form were thought most perfect 
and were applied to all endeavors, including 
engineering, art and architecture. 


Da Vinci, who sketched this canon of 
proportions about 1500 A.D., adhered to a 
description by Vitruvius IfOO years earlier 
Courtesy of the Science Museum, Boston, Mass. 
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the “toise”, however, precedes this by a 
thousand years. 

EARLY STANDARDS 
IN ENGLAND 

English standards can be traced as far 
back as ancient Egypt and Mesopotamia. 
Roman influence had an impact on England 
in both a direct and indirect sense: directly, 
through Roman occupation, and indirectly 
through the Saxon invaders who used the 
Roman system of weights and measures. 
The Saxons brought to England such units 
as the “foot,” or one-half cubit, in 410 a.d. 

The foot is recorded by the Romans to 
have been in official use in lower Germany 
by 12 B.c. 

Succeeding sovereigns, William I (1066) 
for example, seem to have maintained ex¬ 
isting Anglo-Saxon weights and measures. 

There are allusions to actual physical 
standards in England as early as the reign 
of King Edgar (958-975). Hallock suggests 
that these were the earliest authoritative 
Anglo-Saxon standards.* They were ap¬ 
parently housed in places of safekeeping 
where copies were dispensed. In the ab¬ 
sence of clear, enforcing authority, it is 
doubtful that these standards were widely 
used. Standardization of units finally oc¬ 
curred, however, with the unification of the 
country. The “Yard of King Henry” 
(1496) a bronze end bar, might be consid¬ 
ered the first official standard of length. 
The “Queen Elizabeth Yard,” another 
kind of end bar, replaced the “Yard of King 
Henry” in 1588, and was in use until 1824. 

Commenting on the crudity of the 
“Queen Elizabeth Yard” as a standard, 
Francis Baily, a metrologist, stated in 
1834: “This curious instrument, of which 
it is impossible, at the present day, to 
speak too much in derision or contempt. 
A common kitchen poker, filed at the ends 

*William Hallock and Herbert T. Wade, Outline 
of Evolution of Weights and Measures and the Metric 
System, p. 38. 


in the rudest manner by the most bungling 
workman, would make as good a standard. 
It has been broken asunder, and the two 
pieces have been dovetailed together, but 
so badly that the joint is nearly as loose as 
a pair of tongs. The date of this fracture I 
could not ascertain, it having occurred 
beyond the memory or knowledge of any 
of the officers of the Exchequer.” Baily 
goes on to say, incredulously, that copies of 
this standard were in circulation throughout 
Europe and America as recently as ten 
years before, and were certified as true 
copies of the English standard* 

In 1824, England decreed the “Yard of 
1760,” a line standard,** to be the official 
standard of length measurement. The 
primary reason this standard was accepted 
is apparently due to a claim by its origi¬ 
nator, Bird. He maintained that it had 
been derived from a pendulum beating 
seconds of mean time at London latitude 
in a vacuum and at sea level. Because it 
was a natural standard, it could be re¬ 
produced according to a specific formula 
if destroyed. This standard of measurement 
represented the desire to equate length 
with some unchanging magnitude in nature. 

THE METRIC SYSTEM 

The quest for a “natural” standard began 
about the same time in France. In the 
1790’s, the “meter”, one element of an 
integrated, decimalized system of mass, 
length and temperature, was introduced. 
This system of measurement came to be 
known as the metric system. 

A scientific approach to measurement 
was evolving. Galileo had discovered the 
law of the pendulum in 1581, and by 1665 
Huyghens devised a pendulum for re¬ 
cording time. In 1671, Picard suggested 

*William Harkness, “The Progress of Science as 
Exemplified in the Art of Weighing and Measur¬ 
ing,” Smithsonian Institution Report, July, 1888, 

p. 601. 

**This standard is actually composed of dots or 
points which show wear from contact with a beam 
compass. 


using the pendulum’s beat as a recoverable 
standard of length. A length standard de¬ 
rived from a seconds pendulum, however, 
was disdained by the French because they 
recognized that gravity was not constant 
geographically. Instead, the meter was de¬ 
termined by years of painstaking survey¬ 
ing and calculation to be equivalent to 
1/10,000,000th the length of the north 
polar quadrant of the Paris meridian (the 
distance between Dunkirk and Barcelona). 
A special platinum end standard, known as 
the “Metre des Archives,” Figs. 151, 152, 
was constructed as the physical embodi¬ 
ment of this mathematical relationship. 

While this was an heroic scientific under¬ 
taking, subsequent geodetic calculations 
proved the meter slightly in error. At any 
rate, it became an impractical basis of 
comparison, since two bars could be 
equated with a greater degree of precision 
than they could be compared to the cir¬ 
cumference of the earth. For this reason, 
the Metre des Archives itself came to be 
regarded as having primary status. 

Attempts to enforce the metric system 
in France caused confusion, even civil dis¬ 
order. As a result, the system was not re¬ 
vived for many years. 

LATER ENGLISH STANDARDS 

England’s “natural” standard, the Yard of 
1760, underwent a tortuous course similar 
to that of the French Metre des Archives. 
It was destroyed in a fire in 1834, and could 
not be reconstructed from Bird’s formula 
which was found to be based on erroneous 
calculations. 

A more practical course was chosen. Sev¬ 
eral copies of the Yard of 1760 (notably 
those belonging to Troughton and Simms, 
London instrument maker) were com¬ 
pared, from which five new line standards 
were constructed. Using a special alloy 
known as “Baily’s Bronze,” two wells, 
36 inches [914.4 mm] on center, were sunk 
in the top surface of the bar. A gold plug 
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was inserted into each of these wells to a 
depth where the upper surface of the plugs 
coincided with the neutral bending plane 
of the bar. The distance between a finely 
scribed target line on the upper face of 
each plug of bar No. 1, at 62° F [16.67° C], 
was defined as the Imperial Standard Yard. 
Fig. 153 shows Baily’s line yard along 
with other Imperial length standards. 

In 1855, Baily’s line yard became the 
official standard and remained so until the 
Weights and Measures Act of 1963. Inter¬ 
estingly, the four other bars, “Parliamen¬ 
tary copies,” all differing in length, were 
each inscribed with the exact temperature 
at which they would equal the length of 
the Imperial Yard when at 62° F [16.67° C], 
rather than their lengths when at 62° F. 

Though the Imperial Yard had under¬ 
gone a secular change in length of .0002 
inches [0.005 mm], suspected by the Na¬ 
tional Physical Laboratory as early as 1932 
and confirmed in 1947, this was, neverthe¬ 
less, by act of Parliament, the Standard of 
the Realm to which all working standards 
had to be calibrated, Fig. 154. 

Metric standards of length were never¬ 
theless maintained, Fig. 155. 

EARLY STANDARDS 
IN AMERICA 

In his first message to Congress, January 8, 
1790, President Washington stated: “Uni¬ 
formity in the currency, weights and mea¬ 
sures of the United States is a subject of 
great importance, and will, I am persuaded, 
be duly attended to.”* 

Curiously, though the authority to fix 
weights and measures was constitutionally 
relegated to Congress, that body has his¬ 
torically failed to legislate. 

Thomas Jefferson’s proposal in 1790 
when he was Secretary of State, to base 
the American units on a decimal system, the 
length standard of which would be derived 

*William Hallock and Herbert T. Wade, Outline 
of Evolution of Weights and Measures and the Metric 
System, p. 111. 




from a seconds pendulum, reflected the de¬ 
sire of the scientific community in the 
United States for some absolute standard. 
No official standard existed in post-revolu¬ 
tionary America. Consequently, standards 
in use seldom agreed. 

In 1798, Eli Whitney was given a con¬ 
tract by the government to produce a 
number of muskets, the components of 
which would be interchangeable. Since 
this probably was the earliest attempt at 
manufacturing to dimensions, the question 
arose as to “whose dimensions?” To be 
sure, Whitney’s own parts were perfectly 
interchangeable; yet components of the 
supposed same length made by another 
manufacturer, Smith, would not fit any of 
Whitney’s muskets. Why? Whitney and 
Smith were not using the same scale (see 
page 146). 

The United States was not to have what 
could be called a legal standard of measure¬ 
ment for some 60 years. Firstly, the Ameri¬ 
can system of weights and measures was 
largely of English derivation, and un¬ 
friendly relations continued with England 
for about 40 years after the War of Independ¬ 
ence. Secondly, the newly-emerged nation 
was reluctant to adopt a sweeping change 
of the kind that caused civil disorder in 
France when it adopted the metric system. 

In response to a request by Congress 
four years earlier, a comprehensive report 
on the subject of weights and measures was 
issued by John Quincy Adams in 1821, 
then Secretary of State. While admiring 
the “uniformity, precision, and “signifi- 
cancy”* of the French metric system, 
Adams regretted its lack of relationship to 
any convenient unit of the human body, 
and overemphasis of decimal values. He 
recommended to Congress adherence to 
the English system with “no innovation”,** 
and even urged closer use of it by acquiring 

* Charles Davies, The Metric System Considered 
with Reference to its Introduction to the United States 
p. 229. 

**Ibid., p. 300. 


a copy of the Standard Yard from the 
Exchequer of Great Britain.* 

In 1830, Congress having failed to act, 
Ferdinand Hassler, a Swiss immigrant- 
engineer and the first Superintendent of 
Coast and Geodetic Survey, proceeded to 
establish his own standard. The length 
standard he had secured in 1814 was an 82- 
inch [2082 mm] scale prepared by Trough- 
ton of England, based on the Yard of 1588, 
the same as that scorned by Baily. The 
distance between the 27th and 63rd inch 
marks on the scale (seemingly most exact) 
were chosen to represent a yard. 

As Perry observes: Setting up shop in 
an arsenal, Hassler began making copies of 
his standard, thenceforth known as the 
Troughton Scale, and issuing them to the 
several states and customs houses, without 
the least authority from Congress. When 
Congress became aware of this usurpation 
of its power, it quickly passed a resolution 
—urging Hassler to hurry!** 

In 1856, the U.S. received, with presi¬ 
dential ceremony, Bronze No. 11, a yard 
standard, which had been calibrated to the 
Imperial Yard of England. Although 
Bronze No. 11 replaced the Troughton 
Scale as a national standard, it, too, was 
never authorized as an official standard. 
*Ibid., p. 303. 

**John Perry, The Story of Standards, p. 70. 


INTERNATIONAL STANDARD 
On May 20, 1875, an International Metric 
Convention was held in Paris. Attending 
nations agreed to jointly sponsor an Inter¬ 
national Bureau of Weights and Measures 
(BIPM) to be located at Sevres, France, 
Fig. 156. 

Since the time of its inception, the BIPM 
has expanded its scope to include numerous 
projects such as nuclear, time, weight, elec¬ 
trical, and gravity (“G”). The total staff of 
the BIPM is unbelievably small, with only 
40 members. Its small organization, how¬ 
ever, belies its tremendous contribution to 
the advancement of science and metrology. 
They have acted as coordinators and impar¬ 
tial arbitrators. In addition, they have 
been influential in such critical matters as 
the choice of light wave emissions of kryp¬ 
ton 86 as the basis for a length standard.* 

At the International Metric Convention, 
it was also agreed that the International 
Bureau would have the responsibility of 
preparing a special set of meter standards, 
Fig. 157. One of these bars, the closest in 
length to the Metre des Archives, was se¬ 
lected as the “International Prototype 
Meter.” This standard was chosen without 
reference to measurements of the earth. 

*Author’s conversation with P. Carre at BIPM, 
October 4, 1968. 



fig. 156 —Le Bureau International des 
Poids et Mesure s [BIPM), sponsored by 
many nations of the world, is located in 
international territory in Sevres, France. 
Courtesy of the BIPM. 
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i'Kf, 157 — The International ! Prototype 
Meier and Kilogram were agreed a port as 
the official standards of length and mass for 
those nations attending the International 
Metric Convention, 1875. The Prototype 


Meter, a plat inmn-ir idiitf/t line standard of 
a “Tresea” design, teas the official standard 
of length until replaced in 10HO by the light 
wave emissions of krypton 86. 

Courtesy of She BIPM. 





The bar, made of 90% platinum, 10% irid¬ 
ium in an “X” or “Tresea” design, has two 
finely scribed lines set apart near its ends 
on a highly polished surface at its neutral 
bending plane. The separation of these 
lines was defined as one meter at 0° C. 
[32° F.], the temperature of melting ice. 
The “line” standard was felt to be less sus¬ 
ceptible to damage than the terminal sur¬ 
faces of an end standard.* 

As a result of the 1875 conference, the 
signatory countries received copies of the 
International Prototype Meter. The U.S., 
a participant, received National Prototype 
Meter No. 27. 

The Mendenhall Order,** issued April 5, 

*T. R. Young of the National Bureau of Stan¬ 
dards attributes reversion to a line standard to the 
fact that measuring microscopes were then being 
perfected. Their use made it possible to determine the 
position of the line with a great deal of precision. 
Author’s conversation at NBS, November 26, 1968. 

**It is evident from his language that Mendenhall 
was a strong advocate of the metric system, then 
almost completely sanctioned in continental Europe, 
and had no doubt that its adoption by the U.S.A. 
was imminent. 


1893 in Bulletin No. 26 of the U.S. Coast 
and Geodetic Survey,* stated that the 
length standards of the U.S. thereafter 
have as their authority N ational Prototype 
Meter No. 27. The value of the inch would 
be derived therefrom by the following rela¬ 
tionship: that 1 inch equalled 25.4000508 
millimeters. 

While the U.S. customarily used the 
English system of unit, the relationship of 
the British Imperial Yard to the meter at 
that time was such that 1 inch equalled 
25.399978 millimeters. In 1922 this rela¬ 
tionship was revised so that 1 inch equalled 
25.399956 millimeters. 

In 1951, Canada re-defined her inch so 
that 1 inch equalled 25.4 millimeters 
(exactly). By so doing, it meant that 
there were three different “inches”—that 
of the U.S.A., England and Canada.** 

As recently as 1959, the English-speak¬ 
ing nations of the world compromised on 
the Canadian inch with its simpler conver¬ 
sion factor. It was longer than the English 
inch by .0000017 inch [0.00004 mm] and 
shorter by .000002 inch [0.00005 mm] than 
the U.S. inch. 

LIGHT WAVES 

-AN ABSOLUTE STANDARD 

The desire for an immutable standard 
dates, undoubtedly, to antiquity. Today 
we can only speculate on the coincidence of 
the ancient Babylonian cubit being equal 
(within two parts in a thousand) to the 
length of the Babylonian seconds pendulum. 
Similar scientific motivations prompted 
Jefferson in 1790 to propose the use of the 
pendulum, for the English to temporarily 
adopt it as a standard in 1824, and for the 
French to relate the Metre des Archives to 
the circumference of the earth. 

*The Office of Weights and Measures was given 
responsibility for standards shortly afterwards. In 
1901, this became known as the National Bureau of 
Standards. 

**Recently, both England and Australia have 
initiated plans to gradually convert from the En¬ 
glish to the metric system of measure. 
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pig, 158- The “Koestcrs-Zeiss 
Interferometer” was in use at the 
Physikolisch-Technieehc BundesansUdt 
(PTB ) Germany by the early 1950'$-. 
Courtesy of the PTB. 


Huyghens suggested the basic wave 
theory of light by 1665. The concept of 
using monochromatic light waves as a nat¬ 
ural standard dates to 1827, when it was 
proposed by Babinet. Interferometric com¬ 
parisons were made in 1892 by an American 
scientist, Michelson, and Benoit of the 
BIPM. They calculated the International 
Prototype Meter as being equivalent to 
1,553,164.13 wavelengths of red cadmium 
light at 29.92 inches [760 mm] of atmo¬ 
spheric pressure at 15°C [59°F]. This cal¬ 
culation has since been found to be accu¬ 
rate to about one part in 10 million. 

Benoit, Fabry and Perot in 1905-6 at the 
International Bureau also made interfero¬ 
metric comparisons of the meter and calcu¬ 
lated one wavelength of the cadmium red 
line to be equal to 2534.825 x 10~ 8 inches 
[6438.4696 x 10 -10 meters]. Perhaps on the 
basis of this work, the International Solar 
Union, in 1907, defined the angstrom, a 
unit used in expressing the length of light 
waves, as being: 

1 wavelength of the cadmium red line 
= 6438.4696 angstroms. 

So promising were the possibilities of 
light wave measurement that at an Inter¬ 
national Conference of Weights and Mea¬ 
sures in 1927, it was proposed that the light 
wave emissions of red cadmium light be con¬ 
sidered a “provisional” standard of length. 

In the years that followed, pioneering 
work using the principles of interferometry 
was done by O’Donnell, of the United 
States, who had worked directly with 
Michelson; by Barrell, of England’s NPL, 
and by Koesters and Engelhard, who devel¬ 
oped the Koesters Interferometer, Fig. 158, 
which was in practical use at an early date 
in the Physikalisch-Technische Bundesan- 
stalt (PTB), the German equivalent of the 
U.S. National Bureau of Standards. 

At an International Convention in 
Paris, held on October 14, 1960, 32 states 
unanimously voted the new standard of 
length to be the light wave emissions of 



krypton 86. It was chosen over mercury 198 
and cadmium because of its purity and 
clarity. 

Fig. 159 shows Dr. E. Engelhard, who 
contributed greatly to the adoption of the 
krypton light source, with a krypton lamp. 

The meter is today defined as “. 

equal to 1,650,763.73 wavelengths in a vac¬ 
uum of the radiation corresponding to the 
transition between the levels: 

2p 10 -5d5 of the krypton 86 atom.”* 

The immediate practical implications of 
this re-definition were small, but the long 
range effects from the viewpoint of the 
metrologist will be considerable. The In- 


*The established relationship of the inch to the 
meter remains unaltered, but no longer refers ulti¬ 
mately to the International Prototype Meter. Ac¬ 
cording to the U.S. National Bureau of Standards, 
one inch is now defined as “41,929.399 wavelengths 
in a vacuum of the reddish-orange radiation cor¬ 
responding to the transition between levels 2pio-5d5 
of the unperturbed atom of krypton 86.” 











fig. 160 —Some metrologists feel that the 
present length standard (krypton 86) 7vitl 
eventually be replaced by the laser (light 
amplification by stimulated emission of 
radiation). The principle of the operation 
of a laser-interferometer is demonstrated in 
this schematic. Courtesy Perkin-Elmer Corp. 
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ternational Prototype Meter of 1890, the 
old standard, while of exceptional quality 
and stability, could not be guaranteed im¬ 
mune to damage or secular change. Also, 
errors of determination when using a 
10 OX microscope amounted to as much as 
0.000008 (8 millionths) of an inch [0.0002 
mm] or one part in 10 million (considerably 
less if a photo-electric microscopewereused). 

The present light wave standard has a 
potential accuracy of determination of per¬ 
haps 1 or 2 parts in 10 9 . On the basis of 
observations made since 1892, metrolo¬ 
gists are convinced that light waves are an 
unchanging magnitude. 

Even so, the present standard may .con¬ 
ceivably be replaced by the laser (Light 
Amplification by Stimulated Emission of 
Radiation), which some metrologists feel 
will make possible increased accuracies 
when measuring to greater lengths. 

Fig. 160 shows a schematic view and 
Fig. 161 a cutaway view of a laser inter¬ 
ferometer. 

For measurement purposes, present tech¬ 
niques utilize the gas laser, and in particu¬ 
lar, the helium-neon gas laser, married with 
a suitably designed interferometer. The 
great advantage of the gas laser compared 
to other light sources is its coherency and 
strongly defined fringe pattern over long 




fig. un . -Cutaway of a laser interferometer. 

Courtesy Perkin-Eimer Corp. 
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fig. i(52 —The helium-neon gas laser has 
already been applied as a practical 
length-measuring loot, such as this 
“Automatic Scale Measuring 
Interferometer" of the National Physical 
Laboratory (NFL) in England. 

Courtesy NPL, Esritish Crown Copyright. 


optical paths. The gas laser is already 
established as a practical length-measuring 
tool in most of the national bureaus, Fig. 162. 

The gas laser wave-length, however, is 
dependent—aside from the conditions of 
the environment—on the adjustment of 
the reflecting mirror “chamber”, which 
acts as an optical resonator for the helium- 
neon atoms. While this may be adjusted 
very precisely by using piezoelectric or 
magnetostrictive mirror mounts, it means 
that the laser is not an “immutable” stan¬ 
dard. In fact, it “drifts” over a period of 
time. For this reason, it is essential that it 
be periodically compared against the stan¬ 
dard line of krypton 86. Engelhard sug¬ 
gests that it may be many years before a 
suitable gas laser can be developed for use 
as a primary standard. Theoretically, it 
could be used now if defined in terms of the 
infrared line; however, Engelhard feels 
that the line should be visible.* 

Young and Strang of the NBS, on the 
other hand, maintain that since krypton is 
itself somewhat of an “artificially” derived 
standard, being an atomic isotope, there 
should be no objection to using an infrared 
spectrum laser under carefully specified 
conditions.** 

Reports indicate efforts by the NBS re¬ 
searchers to achieve stable laser wave¬ 
length output based on “saturated absorp¬ 
tion in methane vapor of radiation from a 
3.39 micrometer helium-neon laser.”*** 

The specified equipment and environ¬ 
ment required to perform length-measure¬ 
ments of reference-caliber standards by light 
wave method (measurements must be per¬ 
formed in a vacuum, or alternatively calcu¬ 
lations made of the refractive index of the 
air surrounding the interferometer), means 


*Author’s conversation with Dr. E. Engelhard at 
PTB, October 2, 1968. 

**Author’s conversation with T. R. Young and 
Arthur Strang at NBS, November 26, 1968. 

***See “Laser Technique Provides Precise Length 
Measurement,” Machine Design, February 20, 1969. 



eter and electronic counting. The counts are punched onto 
paper tape which is processed by a computer to give a table 
of errors for the scale intervals.” 



FIG. 163 — Ordinary while, or clear light, 
consists of a blend of different colored rays , 
made visible to the eye by passing light- 
through a prism. 


Ill 






FIG. 104 — A widely-accepted theory 
maintains that tight ray* have undulations 
or leaves (; top )■ According to this theory, if a 
portion of a ray is made to travel a greater 
distance and then is returned to the same 


path, "interference fringes” arc seen as the 
ware crest of one falls into the trough of 
another. An interference fringe thus has 
one-half the value of a light wave (flower). 



Monochromatic light (normal incidence) 
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“Diagram and caption presents a simplified mechanical analy¬ 
sis of WHAT happens when bands form between an optical 
flat and a reflective work piece under monochromatic light, for 
a basis of interpretation of work flatness using interference 
band patterns. (All academic-scientific details explaining the 
WHY of the band formation have been intentionally omitted.) 
“Certain rays, B, D, F, etc., fail where wedge thicknesses are 
just one, two, three, etc., ha/f wavelengths, and are reflected 
partly from the flat and partly from the work. At reflection each 
of these particular rays interferes with itself, in accordance 
with optical laws, thus cancelling its own light and appearing 
from above as a narrow dark band. Since each dark band is 
like a contour line on a topographic map, it defines a path 
across the wedge wherever its thickness is exactly uniform. 
The dark bands are thus useful for precise measurement of 
work flatness. Other rays, A, C, E, etc., reflect their light up¬ 
ward without interference and appear as wide alternate bright 
bands.” 



fig. ms—Interference band, (or fringe) 
formation. 

Courtesy J, Kenneth Emory, Van Keurcn Co. 


PIO. i go —The phenomenon of fringe 
formation caused by the interference of light 
wares can be applied to the measurement 
of length as well as to the measurement of 
surface flatness of a workpiece, such as 
this Spin Table top. 
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that work in this direction will be carried on 
largely in the bureaus for some years to 
come. 


Light Wave Measurement Described 

Ordinary white or clear light is a blend of 
varied colored rays, the latter sometimes 
made visible in nature as in a rainbow or a 
soap bubble, Fig. 163. Each of these rays 
has undulations, or “waves” of a specific 
length, such as 1/50,000 inch [one-half of a 
pm], from crest to crest. When a portion 
of a ray is made to travel a greater distance 
and then is returned to the same path, the 
phenomenon of “interference fringes” is 
seen as the wave crest of one falls into the 
trough of another, Fig. 164. 

The phenomenon of fringe-formation 
caused by interference of light has long been 
applied as a practical method to measure 
surface flatness. Measurement is made by 
using an optical flat and light source. An ex¬ 
planation is included here as an aid to gen¬ 
eral understanding. In Fig. 165, an analysis 
of this process is given by J. Kenneth Emery. 
Fig. 166 shows a practical application, the 
inspection of a Spin-Table top for flatness 
through use of a monochromatic light source 
and a 12-inch [305 mm] master optical flat. 

Once the value of a fringe is known, pro¬ 
viding one is able to compute their number, 
or fractions, the metrologist has a conve¬ 
nient, versatile, and precise length-measur¬ 
ing tool. Since white light is a mixture of 
rays, it can be said to have only a mean 
wavelength; its use is therefore limited to 
extremely small distances. The slight differ¬ 
ence in the accumulated length of the wave 
of each ray quickly becomes significant, 
obscuring the fringes.* 

The solution is in the use of monochro¬ 
matic (single color) light. The practical dif¬ 
ficulty is in obtaining a light source which 
will emit monochromatic light of sufficient 

*Fizeau used white light in the first accurate de¬ 
terminations of the coefficients of expansion of 
various materials—but over extremely small ranges. 



Telescope 


Monochromatic light from source S is arranged to pass 
through collimating lens L, after which the light falls on sur¬ 
face A, which has a thin film of silver so as to be semi- 
reflecting. 

The light at A is divided into 2 beams; call them beam 1 and 
beam 2. 

Beam 1, after passing through glass plate P is reflected at F, 
mirror R, to F, mirror N; from whence beam 1 is reflected 
back through the same path, through glass plate P to interior 
of semi-reflecting surface where it is reflected through glass 
plate P to be viewed at telescope T. 

Beam 2, after reflection at A is made to pass through glass 
plate Q, which is of the same thickness as glass plate P. It is 
next reflected at B of mirror M back through the same path, 
through semi-reflecting surface A to be re-combined with beam 
1 at A , again to be viewed at telescope T. 

If, however, there is any difference in the length of the paths 
of beam 1 (ADFEFDAC) and beam 2 (ABAC), interference 
fringes result where the two beams combine, to be viewed in 
telescope T. 

Professor Michelson measured the small path differences by 
displacing one of the mirrors a slight amount by means of a 
micrometer screw. 

Using 3 kinds of light from the cadmium spectrum—red, green 
and blue, he was able to determine the length of each wave 
and also the number of times that each was contained in the 
standard meter. 



In phase Out of phase 

(Fringe shown every 
half-wavelength.) 


fig . \Cu --Thc first true light-ware 
measurements were performed hy Michelson 
and Benoit, -using red cadmium light, at 
the International Bureau in 1892. 
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FIG. 1.88 —Six physical length standards, 
each different inform. Physical standards 
of length of reference caliber are either ( 1 ) 
end standards or (2) line standards. 
Courtesy NPL, British Crown Copyright. 



Modern end standard 




International prototype metre 



Elizabeth yard 


Henry VII yard 





PRECISION SCALE 



purity and clarity for measurements of 
length up to one meter. Fig. 167 shows the 
arrangement of a Michelson type of inter¬ 
ferometer. 

2. Physical Standards as 
Measured in the Bureaus 

While we now have a “natural” standard, 
the need for physical standards remains. 
Moreover, the length of these physical stan¬ 
dards must still be accurately established. 
They are basically in two forms, Fig. 168: 
I. Precision scales 
II. End standards 

I. THE PRECISION SCALE 

The Precision Scale of reference-standard 
caliber is usually of a Tresca, or sometimes 
an “H” section, designed to be supported 
and used horizontally, Fig. 169. Fine lines 
are engraved 90° to the longitudinal axis of 
the bar, on a highly polished, flat surface at 
the neutral bending plane of the bar, Fig. 
170. A convenient line spacing would be.050 
inch in the English system and 1 millimeter 
in the metric; but it is the overall length, 
notably of meter and yard standards, which 
is of particular interest to the bureaus. The 
gaging length is a function of the separation 
of these lines. To minimize the error due to 
out-of-parallelism of the lines themselves, 
only a certain portion of the line is used. 

Engraving the Scale 

A specially-constructed ruling engine is used 
for engraving the lines of the scale. Its spac¬ 
ing is governed by a precise, compensated 
lead screw, or alternately by reference to a 
master scale, using either an optical micro¬ 
scope or a photoelectric microscope. 

Lines are scribed with a sharp diamond 
cutter to widths as small as 80 millionths of 
an inch [0.002 mm]. The trend in recent 
years has been to make these lines as fine 
as possible and to employ higher magnifica¬ 
tions in reading the lines. Precautions are 


FIG. loo— Line standards or precision 
scales, of reference quality, are usually of 
an “I!” or “Tresca” design for rigidity. 


fig. 170 —'The lines of a reference caliber 
precision scale are placed on the neutral 
bending plane to minimize errors due !$ 
flexure of the bar. A convenient line 
spacing is .050 in. in the inch system and 
one millimeter in the metric system. 
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fig. 171 —Until i960, the official standards 
of length were in the form of line standards. 
End standards ultimately had to refer to 
existing line standards. This method of 
comparison was perfected by the NPL. 
Courtesy NPL., British Crown Copyright, 


taken to produce lines of uniform thickness, 
parellelism and spacing. In order to achieve 
isolation from vibration, the entire ruling 
engine is usually mounted on a massive con¬ 
crete foundation which is, in turn, mounted 
on springs. Temperature must be held to 
exactly 20° C [68° F] throughout the ruling 
period. 

Method of Comparison 

Inspection of the scale may be done by com¬ 
parison to another reference scale. Calibra¬ 
tions of the highest order, however, can be 
accomplished through use of a Michelson- 
type of interferometer. The scale is mounted 
on a movable carriage. The mirror is fixed 
to one end of the scale, or to the carriage. 
As the carriage is moved, the lines of the 
scale are picked up by photoelectric read¬ 
out, but the actual displacement is mea¬ 
sured in terms of the number of wave¬ 
lengths of light movement of the mirror. 

II. THE END STANDARD 

The end standard is usually rectangular or 
circular in section. Smaller sizes, such as the 
familiar gage blocks, are essentially used 
vertically, while the larger sizes are most 
often designed to be used horizontally. It is 
the terminal surfaces of the gage, in any 
case, which are the gaging surfaces. 

Lapping of the gage is a cut-and-try pro¬ 
cess. Starting oversize, the ends of the stan¬ 
dard are lapped progressively closer to exact 
size, but consideration is also given to micro¬ 
finish, flatness, and parallelism of the ends. 
Temperature is not quite so critical during 
the lapping operation itself, as in the many 
intermediary inspections of the gage when 
the result of each lapping operation is de¬ 
termined. 

Method of Comparison 

Since the official standards of length were, 
until 1960, in the form of a precision scale, 
akin to the International Prototype Meter, 
many methods of comparing the end stan- 
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dard to the scale had to be devised. One 
metrological procedure involved wringing 
line-ruled block gages to the ends of the 
standard, and reversing the blocks to aver¬ 
age their error, Fig. 171. The end standard 
can be measured more straightforwardly 
by comparison to another end standard, 
either by interferometric means, or by a 
comparator, such as the NPL’s “Level 
Comparator,” Fig. 172. 

Highest order of accuracy, however, is ob¬ 
tained by measuring directly in terms of 
wavelengths of light. Terminal surfaces of 
the end standard, if near-perfect optically, 
can be used as interferometer mirrors. With 
the Koesters-Zeiss interferometer, for exam¬ 
ple, a larger flat block, serving as a reflect¬ 
ing mirror, is wrung to one end to make the 
ends “face the same way.” 

RELATIVE MERIT OF THE 
PRECISION SCALE AND THE 
END STANDARD 

Subdividing 

One obvious advantage of the line standard 
is that one bar can be subdivided into very 
small increments (such as .050 inch or 
1 mm). This is not possible with the end 



FIG. 172-. This “level comparator” was 

designed by the NPL, and enables the 
comparison of an end standard to a master 
end standard of known length: 

Courtesy of NPL, British Cr#wn Copyright. 
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standard, nor is it desirable to simulate this 
by wringing a large number of blocks to¬ 
gether. 

In the greater lengths (over one yard or 
one meter), fabrication of the end standard 
becomes increasingly difficult. Moreover, 
this method, since it is restricted to de¬ 
terminations of only one size, is costly in 
relation to its limited use. 

Problem of Making to Exact Length 

Metrologists stress the importance of hav¬ 
ing a reference standard of such quality that 
an exact determination of length can be 
made. They perhaps do not place sufficient 
stress on the importance of making the stan¬ 
dard to an exact nominal length (say 18 
inches ±5 millionths of an inch [457 mm 
± 0.000127 mm]. The closer to exact length, 
the less is the influence of the comparators 
which must transfer its length to derived 
standards; also, the more fully does the stan¬ 
dard fit the needs of science and industry. 

The end standard can, in fact, be made 


in the greater lengths to within a few mil¬ 
lionths of an inch [small fraction of a pm] 
of nominal size, though the precision scale 
may depart one-ten thousandths of an inch 
[0.0025 mm] or more from nominal. The 
reason for this disparity is that the end 
standard may be lapped progressively closer 
to nominal and as much time as required 
can be spent in inspection to determine 
what slight corrections should be made. 
Once graduated, the scale cannot be altered. 
Many factors could cause the line to depart 
from its prescribed position, including: 

1. Error of the standard or error in read¬ 
ing the standard. 

2. Deflection of the diamond in cutting. 

3. Uniformity of line width. 

4. Parallelism of lines. 

5. Temperature (must be exactly 68° F 
[20° C]). 

6. The reference standard and the stan¬ 
dard to be ruled must be exactly the 
same temperature. 

It should be noted that the above conditions 
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must be exactly controlled during the instant 
of engraving. 

Magnification 

The ability to measure closely depends to a 
large extent on how much the error can be 
magnified. The end standard can be mea¬ 
sured with comparators (such as the elec¬ 
tronic indicator) of 100,000 to 200,000 mag¬ 
nification. The precision scale, on the other 
hand, can be read by microscopes limited to 
50 or perhaps 100 power and then only 
where line quality is exceptional. The rea¬ 
son for this vast difference in magnification 
is that the terminal face of the end standard 
may be considered an infinitely thin line in 
space. The graduation on the line standard 
is a groove, which is a minimum of 80 mil¬ 
lionths of an inch [0.002 mm] wide. 100X is 
the approximate limit of magnification be¬ 
cause of lens distortion. At 100,000 power, 
the line would be 8 inches [203 mm] wide, 
and the line begins to look something like a 
rutted country road. 

To overcome this limit of magnification 
when using an optical microscope, photo¬ 
electric microscopes of 10,000 times res¬ 
olution have been developed. They are 
used with line standards whose quality and 
purpose justify their use. It is estimated 
that accuracy of determination increases 50 
times. 

There appears to be some difference, how¬ 
ever, between what the photocell registers 
in picking up the line and how the human 
eye, using an optical microscope, interprets 
this same 80 millionths of an inch [0.002 
mm] as a target. The scale, or any of its 
derivatives, calibrated photoelectrically, 
may thus have some other calibration when 
read by eye, Fig. 173. 

RELATION TO THE 
INTERNATIONAL STANDARD 
-LIGHT WAVES 

The end standard as a physical representa¬ 
tive can be directly compared to wavelengths 


fig. 173—This chart was prepared on the 
basis of a controlled test conducted by the 
NPL. hi the test, one particular precision 
scale was calibrated, first by referencing the 
line by optical microscope (visual), and 


then by photoelectric microscope 
0 automatic'). 'The difference in the 
calibration amounted to ±0.3 pm 
[±12 millionths of an inch]. 
Courtesy NPL, British Crown Copyright. 
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of light by absolute means (no mechanical 
movement, no time lag). This is accom¬ 
plished by using its terminal surfaces as 
optical reflectors in the interferometric 
arrangement. 

The line standard, on the other hand, is 
compared indirectly to the light wave stan¬ 
dard through movement of a carriage hold¬ 
ing the mirror. The line is picked up opti¬ 
cally or photoelectrically. A fringe-counting 
interferometer, which takes a signal and 
commits it to a photo-sensitive cell, is re¬ 
quired. The problem is to hold temperature 
drift, since there is a time interval while 
counting fringes. In addition, there is the er¬ 
ror of the carriage itself such as to straight¬ 
ness of travel. Line standards thus cannot 
be compared to the official light wave stan¬ 
dard by absolute means. 

The techniques of relating the end stan¬ 
dard and the precision scale to the krypton 
standard are compared in Fig. 174. 

The problem of wear was considered suf¬ 
ficiently worrisome that the International 
Prototype M eter, unlike its predecessor, the 
Metre des Archives (see page 105), was in 
the form of a precision scale. This is not 
now as significant a factor, since the ultimate 
standard is no longer in physical form, and 
is reproducible. 

CONDITIONS OF COMPARISON 

Determinations of length must be made 
under certain specified conditions. The ac¬ 
curacy of determination is in direct propor¬ 
tion to how well these conditions are met 
and how rigidly maintained. 

Environment 

Elements within a standards room environ¬ 
ment which may influence the comparison 
of linear standards include temperature, 
atmospheric pressure, humidity and vibra¬ 
tion. 

Temperature 

The reader will recall that non-uniform tem¬ 
perature within a piece may cause distor- 





fig. 174 — The techniques of relating the line 
standard and the end standard to the 
krypton light standard are compared. 

The end standard may be compared to 
the light wave standard “absolutely ” with no 


movement (bottom). The comparison of the 
precision scale to the light leave standard 
necessitates the movement of a carriage, 
supporting the standard—nol absolute 
(top). 
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fig. 175 —Various materials expand or 
contract at different rates in response to 
temperature change. Tables of coefficients of 
expansion are less exact than would be 
desired, especially for the purpose of 
metrology. 


COEFFICIENT OF THERMAL EXPANSION 
OF DIFFERENT MATERIALS 


Aluminum 
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Linear expansion 
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per degree F 


tion. Measurements of length involve two 
additional considerations: 

1. Virtually all bodies will expand with, an 
increase in their temperatures. The re¬ 
verse is also true.* It is meaningless, 
then, to attempt to state the exact 
length of an object without at once 
specifying the temperature at which 
this is so. 

2. Rates of thermal expansion vary with 
different materials. For this reason, 
tables of “coefficient of thermal ex¬ 
pansion” have been formulated for 
different materials. They enable cal¬ 
culation of the amount of expected lin¬ 
ear change in a body, given its length 
and temperature change, Fig. 175. 
These tables, unfortunately, are only 
theoretically correct. If the composi¬ 
tion is even slightly altered or is non- 
uniform, the stated coefficients may 
not apply. Moreover, the stated coef¬ 
ficients refer only to a change of one 
degree, from 0° to 1°C [32° to 33.8°F]. 

Onewould expect, then, that some specific 
temperature must be chosen at which all 
calculations of length are to be made. En¬ 
gland, for example, used 62°F [16.67° C] for 
many years. Also the meter was originally 
defined at 0°C [32° F], the temperature of 
melting ice. 

Today the internationally-accepted tem¬ 
perature at which length determinations 
are to be made is 68° F [20° C]. The choice 
of 68° F, while arbitrary, is convenient 
enough. At this temperature, there is a con¬ 
venient numerical conversion between de¬ 
grees Fahrenheit and degrees Celsius (Cen¬ 
tigrade). Most importantly, everyone must 
agree to and adhere to this figure. 

If the length standards have identical co¬ 
efficients of expansion, it follows that when 
being compared they need only be alike in 
temperature. However, the more dissimilar 
the coefficient of expansion of the standards 


*This excludes temperature variations wide enough 
to cause secular changes or changes in state. 


being compared, the more necessary it is 
that the temperature of both be exactly 
68°F [20°C], 

While corrections can be made to the ex¬ 
tent temperature is not exactly maintained, 
the goal should be to eliminate or at least 
minimize departures from ambient. Ambi¬ 
ent temperature change has three sources: 
convection, conduction and radiation. 

Convection, Fig. 176. Ambient tempera¬ 
ture change of the standard due to convec¬ 
tion might include the effects of opening 
doors or windows, drafts from the move¬ 
ment of personnel, or variations from the 
temperature-control unit itself in the form 
of non-uniform air currents, or even strati¬ 
fication within the room. 

Conduction, Fig. 177. Temperature dif¬ 
ferentials may also be caused by conduction, 
the flow of heat within a body or from one 
body to another. Thus, the standard may 
itself depart from ambient when in contact 
with some object above or below ambient. 
An example of this would be the handling 
of the standard by personnel. 

An unsuspected ambient error can arise 
from stratified temperature within the room. 
The surface on which the standard rests 
(such as a surface plate or machine table) 
may itself be exposed to convective air cur¬ 
rents above or below 68°F [20° C] at some 
other level of the room. 
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Via. 170 —A badly-designed temperature- 
control system may cause greater errors than 
the absence of any controls at all, especially 
if the controlled air exhausts near machines 
or gaging areas. Errors are caused by 
convection. 


The principle of conduction, however, 
can be utilized to achieve stability. A sur¬ 
face plate or machine table of sufficient mass 
held to68°F [20° C] can act as a “heat sink.” 
Placing the standards in close proximity, or, 
better still, in contact with one another on 
such a surface contributes to conformity of 
their temperatures. 

Radiation. The effects of radiant energy 
are especially troublesome, and often over¬ 
looked, where a high order of accuracy is to 
be attained. All objects in a given room en¬ 
vironment emit and absorb energy in the 
form of electromagnetic waves traveling in 
a straight line, tending towards an equilib¬ 
rium of temperatures. However, a body 
above ambient placed within this environ¬ 
ment raises the temperature of other objects 
to the degree in which they are in the path 
of these waves, and also depending on their 
surface color and texture. A dark surface 
tends to absorb heat; a shiny surface to re¬ 
flect. For example, due to absorption of 
radiant energy an unshielded gray granite 
surface plate will be 0.1 to 0.2°F. [0.055 to 
0.11°C] above room temperature and a 
black granite plate as much as 0.4°F 
[0.2° C], Fig. 178. 

Sources of radiant heat include lights, 
sunlight, personnel, electrical equipment, 
heaters or large masses above ambient 
brought into the environment. Aluminum 
foil or other guards may be used to shield 
the measurement set-up from radiant heat. 
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fig. 177 —(center) The handling of standards 
by personnel may alter their lengths 
through conduction. 

fig. ns—(bottom) Objects in the direct path 
of light will vary in temperature depending 
on their rates of absorption or reflectivity of 
that light. Errors are caused by radiation. 
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FIG. 1 70 — An above-ambient standard 
brought into a controlled environment, 
requires several hours to reach ambient, 



Hours 


Temperature and Time 
When the standard is allowed to depart 
from ambient temperature, whether due to 
handling, exposure to radiant heat or tem¬ 
porary removal from the controlled area, 
there is a time lag, depending on the 
thermal capacity of the standard, before 
it again reaches ambient. 

Plotting the temperature of the standard 
against time, the curve is exponential, Fig. 
179. It should be noted that an unsuspected 
temperature differential can exist for many 
hours after the source of heat is removed. 

If the standard is placed with as much of 
its surface as possible in contact with a large 
mass at ambient within the room, the time 
required for the temperature differential to 
disappear lessens. Even so, where a few mil¬ 
lionths of an inch accuracy [tenth of pm] is 
required, it is recommended to wait many 
additional hours to allow the standard 
to reach ambient. 

Humidity 

The presence of humidity within the mea¬ 
suring area is not apt to dimensionally 
change metallic standards which are most 
commonly used. Its effect, however, should 
not be completely discounted. Various non- 
metallic substances are seriously effected 
dimensionally by humidity, and humidity 
will corrode many ferrous materials. 

Vibration 

Vibration, as with humidity, has no di¬ 
mensional effect on the standard itself. Suf¬ 
ficient amplitude of vibration might cause 
a shift of the standard’s position during 
the calibration. The presence of even the 
slightest vibration, however, can definitely 
frustrate the use of high magnification 
instruments, notably interferometers, and 
to a lesser extent mechanical and elec¬ 
tronic indicators. 

Atmospheric Pressure 

Atmospheric pressure is not usually con- 
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sidered a factor in calibrations of length, ex¬ 
cept where comparisons are made interfero- 
metrically. Measurements are then either 
made in a vacuum or compensations are 
made which include corrections for the re¬ 
fractive index of the air. 

When gages are measured in a vacuum, 
corrections must be made for the compres¬ 
sion that occurs on return to normal atmo¬ 
spheric pressure. For a one-inch [25.4 mm] 
gage of steel, this amounts to one-third of a 
millionth of an inch [0.000008 mm].* This 
might seem a small amount until one con¬ 
siders that the value for a meter standard 
[39.37 inches] would be approximately one- 
third of a fim [13 millionths of an inch]. 

Engelhard performs many interfero¬ 
metric length measurements without vac¬ 
uum, feeling that there may be less potential 
error in calculating the refractive index of 
the air than in compensating for the return 
to normal atmospheric pressure.** 

Material of the Standard 

Stability— Nothing is more disturbing, 
metrologically, than dimensional instability 
of the measuring standard. One comes to 
trust the standard as calibrated, when in 
fact it has undergone some long or short- 
run change in length. It might be remem¬ 
bered that the Imperial Yard underwent at 
least one modification in its length due to 
secular change. 

Residual stresses, such as those from 
machining, may be later evidenced by a 
bending or twisting of the standard, altering 
its effective length. 

End standards are usually hardened to 
facilitate lapping of their gaging surfaces, to 
minimize damage from being bumped, and 
to reduce wear in use. Usually, only surface 
hardness is required, the core remaining soft 
for greater stability. At times, only the ter¬ 
minal or support surfaces are hardened. 

*Irvine C. Gardner, “Ten-millionths of an inch,’’ 
Ordnance, May-June 1965. 

**Conversation with Dr. E. Engelhard at PTB, 
Oct. 2, 1968. 


Precision, hardened steelgageblocks may 
undergo astonishing changes in length with 
improper heat treatment. The presence of 
unstable austenite retained from the hard¬ 
ening cycle, or untempered martensite, may 
result in a crystalline phase change. This 
may either occur slowly over a period of 
time, or be suddenly triggered by a jar or 
variation of temperature. 

Experiments by Meyerson and Sola* 
make clear that there remains much to be 
learned about dimensional stability. In¬ 
cluded in their tests were many types of 
steels, such as 52100 modified, nitrided 410 
stainless, and other materials, such as cer¬ 
mets and ceramics. Many of these materials 
were also analyzed for stability after subjec¬ 
tion to varying treatments. Their experi¬ 
ments were not conclusive in all respects, yet 
the following groundwork was established: 

1. Several materials, with prescribed 
treatment can be made ultra-stable 
(no more than .1 or .2 millionth inch/ 
inch/year change), [0.0000025 or 
0.0000050 mm/25.4mm/year[. 

2. Given the requirement of hardness of 
65Rc or greater, for strength and wear, 
then the dimensional stability of the 
best surface-hardened steel will exceed 
that of any of the through-hardening 
steels. 

3. The hard case of surface-hardened 
steels should not be removed, even 
from the sides. This applies to case- 
hardened as well as to nitrided speci¬ 
mens. The case apparently acts as a 
“protective envelope,” minimizing 
damage and resisting stress from 
bumps or shocks. Meyerson and Sola 
hold to the interesting theory that the 
case also acts as a “restraining jacket,” 
to eliminate any movement of the 
core. In the 410 stainless series, how¬ 
ever, it is important that the thin 

*Melvin R. Meyerson and Marcos C. Sola, “Gage 
Blocks of Superior Stability III: The Attainment of 
Ultrastability,” ASM Transaction Quarterly, March 
1, 1964. 


white oxide film be removed from all 
surfaces of the specimen after being 
nitrided and hardened. 

4. Metallurgical composition effects sta¬ 
bility. For example, two titanium car¬ 
bide blocks with nickel binders were 
tested. One with 25 % nickel was stable, 
while the 40% nickel block was not. 

5. Surface treatments, such as thermal 
spraying, chromium plating and flame 
plating were generally of inferior sta¬ 
bility; if plating did not adhere prop¬ 
erly, even greater instability resulted. 

Of note is the fact that nitrided and hard¬ 
ened “Nitralloy” was found to be among 
the ultra-stable materials, and is generally 
recommended for heavy-duty applications 
(see page 187). 

The choice of materials and methods for 
fabrication and heat treatment must be 
made to assure maximum stability. Surely 
the diligence required to attain accuracy 
should be rewarded by some assurance of 
its permanence. 

Coefficient of Expansion —Nickel-steel al¬ 
loys have an interesting characteristic. As 
the percentage of nickel added to steel in¬ 
creases, the thermal coefficient of expansion 
drops rapidly. A minimum is reached at 
36 % nickel, where the coefficient is approxi¬ 
mately one-sixth that of steel (see Table, 
Fig. 175). This alloy, Invar, was used by the 
national bureaus for some of their linear 
standards until found slightly unstable. 

The choice of material for use as a linear 
standard is widely misunderstood. It is 
often supposed that the material must, like 
Invar, have a low rate of thermal expan¬ 
sion. The apparent assumption is that if 
temperature changes do not much alter the 
length of the standard, then temperature is 
eliminated as a source of inaccuracy. 

Unfortunately, though, when such a stan¬ 
dard is called upon to measure other stan¬ 
dards and gages, usually of steel, we see 
that, in regard to temperature, we have lost 
rather than gained. Standards having dis- 
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pig. iso--Two steel blocks, the some length 
at 68° F [20°Cl, will be virtually the same 
length at 68.1 °F [20.05°C]. 


Steel 68.1 : Steel 

F 


fig. 181 —An Invar block and a steel block, 
ike same length at 68°F [W°C\ will not be 
the same length at 68.1°F [20.05°C\. 
Although this effect may be slight with 
smaller blocks, the disparity is significant 
with greater length. 
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similar coefficients will only agree in length 
at one temperature—68°F [20°C]. When 
standards being compared have widely dif¬ 
ferent coefficients, such as Invar to most 
steels, the slightest thermal uncertainty 
may result in sizeable errors of determina¬ 
tion. Where two linear standards being 
compared have approximately the same 
coefficient of expansion, uncertainty is mini¬ 
mized. It is only necessary that they be as 
close as possible to the same temperature 
and reasonably close to 68°F [20°C], Figs. 
180 and 181. 

Precision scales for this reason are most 
often of 58% nickel steel, which is non- 
tarnishable and has approximately the same 
coefficient of most types of steel. The same 
considerations govern the choice of material 
for an end standard. 

Logically, then, we would have to say 
that the new light wave standard is not nec¬ 
essarily an improvement over the old physi¬ 
cal standard as to its response to tempera¬ 
ture change. Worse than Invar, light waves 
have zero thermal coefficient. Temperature 
is now more critical than it ever was when 
we compared physical standards. The accu¬ 
racy with which we transfer the light wave 
standard to actual physical standards may 
be seriously limited by our ability to main¬ 
tain and measure temperature, as well as by 
our insufficient knowledge of the coefficients 
of expansion of different materials. 

Support of Length Standards 
Relatively small standards, such as gage 
blocks less than three inches [76 mm] in 
length, undergo no appreciable alteration in 
length regardless of the method of support. 

A long standard, if supported vertically, 
will undergo a significant foreshortening of 
length due to gravity (compression due to 
its own weight), Fig. 182. The amount of 
foreshortening increases with length, and is 


fig. 182 —A 16-inch [406 mm] length 
standard undergoes a foreshortening of 
approximately 1.2 millionths inch 
[( 0.00003 mm] from the compression of its 
own weight when supported vertically. 
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calculated as: 



where: 


X = foreshortening 
S = density 
L = length 

E = Young’s Modulus 
of elasticity 

It is preferable to support long standards 
horizontally rather than vertically.* If the 
former, then the method of support is almost 
solely determined by the effects that gravity 
may have in changing its measured length. 

Placing the lines of a precision scale at its 
neutral bending plane minimizes errors due 
to flexure of the bar when used on a surface 
which is not a true plane, Fig. 183. 

However, if the line standard is sup¬ 
ported on the “Airy points,’ ’ ** an absolutely 
true plane is not required and the effect of 
gravity can be virtually discounted. The 
surface at the end of the bar will be in a 
horizontal plane and the separation of the 
lines will have its maximum value. 

Similarly, an end standard, Fig. 184, 
when supported horizontally at the Airy 
points would have virtually the same length 
as when supported vertically (discounting 
compression), and its end faces, if parallel 


*In a Test Report (No. 2.4/158197) by the NBS, 
one of a series of measurements of a 16-inch Moore 
End Standard involved its measurement in a ver¬ 
tical position, and a “. . . correction factor of 
.0000012 (1.2 millionths) inch was applied . . . .” 

Deriving the correction factor: 


^ _ .2818 (16 in.) 2 
X ~2 (30,000,000) 

(approximate Y oung’s 
Modulus for steel) 
or 

X = .0000012 in. 

**The “Airy points” are calculated to be equi¬ 
distant from the ends, and separated by the factor 
.577L, where L is the length of the bar—a relation¬ 
ship arrived at by Sir George Airy while Astronomer 
Royal. 
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fig. iS3 [left), fig. i.S'i (right)—Examples 
of elastic deflection which occur in length 
standards, attributed to the method of 
support. Whether the gage is an end 
standard or a line standard, effective length 
is altered when support is at the ends or the 
middle. In the case of the line standard, it 


is evident that two errors occur: The first is 
a foreshortening of overall length from 
bending; the other can occur if the lines are 
placed on the upper surface of the bar, 
rather than on the neutral bending plane. 
Support on Airy points [.B77L] virtually 
eliminates error from bending. 




FIG. s S6 —A surface trace of a precision 
gage block at one million times rertieal 
magnification. Can its true surface be 
defined? 

Courtesy of E3enchx, Miciometrical Division. 



when vertical, would be parallel when hori¬ 
zontal. 

Bending is also prevented if both the 
standard and the surface on which it rests 
are absolutely flat, a method used with the 
Step Gage (see pages 184-185), where over¬ 
all considerations dictate such an approach. 

Micro-Geometry: Its Relationship 
to the Measurement of Length 

Geometry vs. Micro-Geometry— Our abil¬ 
ity to perform all manner of measurements 
to closer tolerances has progressed at a truly 
rapid rate. Within one man’s life span, the 
standard level of precision has gone from 
“thousandths,” to “ten-thousandths,” to 
“millionths” of an inch [hundredths of a mil¬ 
limeter to several pm to small fractions of 
a pm]. For certain measurements, fractions 
of a millionth inch [hundredths of a pm] 
are now given serious consideration. In fact, 
an avowed goal of the National Bureau 
of Standards is to measure to the tenth part 
of a millionth of an inch [0.0000025 mm]!* 

The reader might ask: “Is there no end of 
the road, no physical limitation to how 
closely we can resolve dimensional values?” 
An answer to such a question was first ad¬ 
vanced by an associate, Frederick C. Vic¬ 
tory, widely recognized for his pioneer work 
in metrology. He wrote many years ago: 

“Accuracy, as a term, must necessarily 
imply degree. Absolute accuracy exists only 
in theory. Progress toward this unattainable 
goal can be compared to that of the frog 
which, starting from the middle of a table, 
coveredhalfthedistancefromhislastposition 
to the edge with each jump, and could thus 
never reach the edge. In strides toward great¬ 
er locational accuracy, the earliest gains were 
most impressive because there was so much 
room for improvement. Now, progress is 
slower, the gains proportionately smallerand 
the problems increasingly perplexing.”** 

*Irvine C. Gardner, “Ten-millionths of an inch,” 
Ordnance, May-June, 1965. 

**Frederick C. Victory and Richard F. Moore, 
Holes, Contours and Surfaces, p. 19. 
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We might also add that closer measure¬ 
ments will always depend on the status of 
many different “arts,” all of which must ad¬ 
vance together. But more specifically, there 
is an obstacle to ultra-fine measurement 
which can be put in the general category of 
“micro-geometry.” 

The term “geometry,” by way of expla¬ 
nation, has been used thus far to describe 
overall shape, such as flatness, parallelism 
and support of masters and machine mem¬ 
bers. It will later be used with regard to 
roundness. 

Gages and standards must also have good 
geometry. The level of perfection attained in 
the flatness and parallelism of terminal sur¬ 
faces of an end standard determines to a large 
extent the “accuracy of determination” 
in calibrations by the national bureaus. 

However, when measurements are to a 
few millionths of an inch [approximately 
1/10 pm], innumerable minute variables, 
previously insignificant, become highly 
critical. With only a few ounces [hundred 
grams or so] of pressure, tips of indicators 
and gaging surfaces squash and bulge to 
millionths of an inch [tenths of a pm]. Sur¬ 
faces which appear normal to the unaided 
eye, when magnified thousands of times, are 
seen to have “mountains, valleys, plateaus 
and craters” millionths of an inch [tenths of 
a pm] in magnitude! By way of illustration, 
Fig. 185 shows a surface trace of a precision 
gage block at 1,000,000 times magnification. 

What can then be called a “true” surface? 
And for that matter, what is geometry? 

Older techniques of measurement and 
more familiar terminology are not always 
adequate for analyzing and discussing the 
micro-inch. The term “micro-geometry” 
best describes the special set of problems 
and conditions having to do with measure¬ 
ments on this level. 

The following analysis is given to indicate 
a major direction for further progress. It is 
moreover aimed at encouraging a more con¬ 
servative attitude in those who make claims 


of working to “millionths”—revealing ex¬ 
actly how small a millionth of an inch actu¬ 
ally is. 

As E. G. Loewen points out, there are 
“confused concepts” with regard to micro¬ 
inch accuracy [tenth of a pm]*. Instru¬ 
ments can detect sensitively minute differ¬ 
ences through high resolutions, but to obtain 
definitive values of measurement is more 
demanding. As an example, he uses the pres¬ 
ent limitations of 1 millionth of an inch 
certainty [0.000025 mm] inthemeasurement 
of gage blocks where conditions approach 
the ideal (two parallel planes). The mea¬ 
surement of the exact diameter of a cylinder 
is infinitely more complex. One aspect of 
this problem will be shown under “Defor¬ 
mation” (see page 129). 

Surface Finish —Perhaps the most per¬ 
plexing consideration involving measure¬ 
ment is that of surface finish, and applies to 
the manufacture of virtually every gage, 
whether optical or mechanical. The senses 
of sight and touch of a qualified inspector 

*Erwin G. Loewen, “Micro-inch Accuracy . . . 
Really?”, American Machinist, June 5, 1967. 


can evaluate surface finish of the average 
specimen with uncanny accuracy. More¬ 
over, for the average specimen, there is gen¬ 
erally agreement on means of measurement 
and definition.Usuallyitseffectson accuracy 
or function are fairly obvious. Aswe shall see, 
however, with measurements of gages hav¬ 
ing a high micro-finish, the effect of surface 
finish is not generally so obvious relative to 
the accuracy sought; and there are even wide 
gulfs of disagreement among leading au¬ 
thorities as to measurement and definition. 

Evolution of Surface Finish Measurement 

Quality control of surface finish at first re¬ 
lied on pure judgment, followed perhaps by 
samples put aside for the workman or gage- 
maker to use as a comparison. Later, sight- 
touch standards (see Fig. 186) were used, 
but provided only rough limits within which 
to work. Moreover, since they relied purely 
on the senses, they did not truly measure. 
Schmalz probably made the first experi¬ 
mental attempts to measure the depths of 
surface irregularities in 1929 through an 
“optical lever.” Both Schmalz and Nicolau, 



FIG. 186 — Sight-touch standards are 
convenient for judging surface finish. Since 
they rely purely on the senses, they do not 
provide measured values, 
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KIG. mi— The “Talymtf” measures surface 
finish by means of a follower stylus. 

Surface characteristics may be magnified 
thousands of times and are graphed through 
use of a linear recorder. 

Courtesy or Rank Industries, Ltd, 


a contemporary, used “grazing lines” on 
graphical charted values as reference lines 
from which to measure—Schmalz from top 
down, Nicolau from bottom up. 

In 1936, Abbot built the first shop instru¬ 
ment, the forerunner of the present “Profi- 
corder” of the Micrometrical Division of 
the Bendix Corporation. This instrument 
employed a moving coil for electronic am¬ 
plification and a supporting skid for a sty¬ 
lus. Because measurements taken with this 
instrument were instead in reference to a 
graphed “mean line” of the profile, a con¬ 
troversy over definition thus began. 

The “Talysurf” of the Taylor-Hobson 
Company, England, Fig. 187, appeared in 
1940. R. E. Reason of that company is today 
perhaps the most outstanding surface finish 
authority. Since that time, many new in¬ 
struments to measure surface finish have 
been developed, but leaders in the field are 
still divided in opinion on how to express 
and measure surface finish. 

Reason points out that . . a system 


which is perfect may be unattainable.”* 
Nevertheless, some common ground and 
language was needed. In 1962, the Ameri¬ 
can Standard Association issued “ASA B 
46.1, Surface Texture,” a technical bulle¬ 
tin which provided classifications for rough¬ 
ness, waviness, lay, and the symbols to be 
used.** The following is an outline (with the 
ASA’s reference number) of some of the 
definitions agreed on, Fig. 188: 

2.1 Surface Texture —“Repetitive or ran¬ 
dom deviations from the nominal 
surface which form the pattern of 
the surface. Surface texture includes 
roughness, waviness, lay and flaws.” 
2.1i Center Line —“ . . . the line about 
which roughness is measured. A 
median line, mathematically.” 

*R. E. Reason, “Report on Reference lines for 
roughness and roundness,” CIRP Annalen, 1962- 
63, pp. 95-101. 

"Information on pages 126-127, including Fig. 
188, is taken from “Surface Texture,” Technical 
Bulletin ASA B 46.1, 1962, with permission of pub¬ 
lisher, The American Society of Mechanical Engi¬ 
neers, New York. 
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2.5 Micro-inch—One millionth in., ab¬ 
breviated MU in. 

2.6 Roughness —“ . . . finer irregularities 
in the surface texture, usually in¬ 
cluding those irregularities which re¬ 
sult from the production process.” 
(feed marks, etc.). 

2.6.1 Roughness Height —“ . . . arithmeti¬ 
cal average (AA) (or CLA Center 
Line Average in British Standard), 
deviation expressed in micro-inches 
measured normal to the center line.” 

2.6.2 Roughness Width— “Predominant 
peak-to-peak pattern of roughness, 
expressed in inches.” 

2.6.3 Roughness-width cutoff —“A selected 
spacing; care must be taken to in¬ 
clude all of the surface irregularities 
to be assessed. Can be as much as 
1.000 inch [25.4 mm] but .030 inch 
[0.75 mm] if no value specified.” 

(The British use “Meter Cutoff,” 
or “sampling length”). 

2.7 Waviness —“ . . . widely spaced com¬ 
ponent of surface texture . . . wider 
spacing than the roughness width 
cutoff” — due to machine or work 
deflections, vibration, chatter, etc. 

2.7.1 Waviness Height —“ . . . rated in 
inches as the peak-to-valley dis¬ 
tance . . . .” 

2.7.2 Waviness Width —“ . . . rated in 
inches as the spacing of successive 
wave peaks or successive wave 
valleys . . . .” 

2.8 Lay —“The direction of the pre¬ 
dominant surface pattern, ordinarily 
determined by the production meth¬ 
od used.” 

2.9 Flaws— “Flaws are irregularities 
which occur at one place or at rela¬ 
tively infrequent or widely varying 
intervals in a surface.” (scratches, 
cracks, random blemishes, etc.). 

3.3 The Surface Symbol 

3.3.1 “The symbol used to designate sur¬ 


face irregularities is the check mark 
with horizontal extension.” V 

Symbols are used to indicate lay of the 
surface pattern. Relative to the surface to 
which the symbol is applied, it may be 
“parallel” II , “perpendicular” JL, angular 
in both directions X, multidirectional M, 
circular relative to center C, or radial 
relative to center R. 

Shop instruments measure all surface 
texture characteristics except “lay”, which 
is observed by eye. 

The ASA Standard would seem to have 
fallen short of perfection. For instance, it 
still has nothing to do with function, and 
does not allow meaningful comparisons be¬ 
tween unlike specimens (milled vs. ground 
for example). It does not include methods 
of measuring other than the stylus-type. 

In retrospect, however, it was probably 
wise to have some limitation of scope, since 
some consensus was obtained. At any rate, 
the stylus-type of measurement is still in 
most general use and is most convenient. 

One limitation of the “arithmetical aver¬ 
aging” system, though, is that it minimizes 
extreme variations which may be signifi¬ 
cant to function. For this reason, although 
it has achieved international acceptance, 
most of the countries that use it admit to 
using supplementary peak-to-valley, or 


RELATION OF SYMBOLS TO SURFACE CHARACTERISTICS 



fig. 188 —An example of terminology 
for surface texture and symbols used is 

shown here. 
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fig. 189—Am interference microscope 
is a valuable means of examining surface 
characteristics. Us advantage is that 
measurement requires no physical contact, 
and is 3-dimensional. 

Courtesy of Carl Zeiss Co. 



D 


“full profile” measurements—something 
of a compromise between the original con¬ 
cepts of Abbot and Schmalz. Considering 
that “full profile” is also employed, the 
claim often made that tip radius is of 
secondary importance is not exactly valid. 

While there is still disagreement among 
the proponents of the stylus system, other 
authorities show that there are funda¬ 
mental limitations to the stylus concept 
itself, in that it is only 2-dimensional.* 
They hold that an evolution to more sophis¬ 
ticated and exacting results will require 
supplementing such measurements with 
micro-interferometry (depth-measured in- 
terferometrically, and the fringe-pattern 
viewed through a microscope), Fig. 189. 
In other words, measurement must be 3- 
dimensional. 

Fig. 190 shows a steel surface in vertical 
light with the comparison path of rays cut 
off, taken with 60-power objective in 
thallium light, X = 0.54pm [0.000021 inch]. 
Total magnification is 480X and the visual 
scale has a diameter of approximately 
0.3 mm [0.012 inch]. Below is Fig. 191, 
showing the same section in the interfer¬ 
ence path of rays. The fringe shift is mea¬ 
sured in fractions of the fringe spacing 
which multiplied by X/2 is the roughness 
height in pm [40 millionths of an inch]. The 
irregularities of the fringes indicate the 
unevenness of the specimen. 

Fig. 192 shows a cylindrical surface. Fig. 
193 shows a spherical surface; the irregular¬ 
ities are caused by a deformation of the ball. 

Some studies show that the tracer-type 
and interference microscope may be in 
fairly close agreement.** Since micro-inter¬ 
ferometry may not be as widely applied, 
the tracer-type is generally the most ac¬ 
cepted method. 

*James B. Bryan et al., “State of the Art Report: 
Measuring Surface Finish,” Mechanical Engineer¬ 
ing, December, 1963, pp. 43-46. 

**F. H. Rolt, "Use of Light Waves for Controlling 
the Accuracy of Block Gauges,” NBS Bulletin 581, 
April 1, 1957, pp. 27-41. 


fig. 190-193 — (a) A. steel surface in vertical 
light with the comparison path of rays cut 
off. Photo is taken with CO-power objective 
in thallium light. X ~-0.5bp.ni. [0.000021 
inch}. Total magnification is b80\; the 


visual scale has a diameter of approximately 
0.3 mm [0.012 inch}. 

(b) The same section of steel surface (see 
Fig. 190) in the interference path of rays. 


(c) A. cylindrical surface under an 
interference microscope. 

(») Irregularities of a spherical surface 
through an interference microscope. 
Courtesy of Car! Zeiss Co. 


128 


















Fro. 1 94 — Local surface deformation occurs 
in the gaging process. In this exaggerated 
example, an indicator probe registers on top 
of a ball. Deformation occurs at 4 places: 

(i) On the surface of the probe (2) At the 


point on the ball where the force of the probe 
is applied ( S) The area of the ball resting 
on the gaging platform, or support 
it,) The surface of the platen in contact 
with the ball. 


When there is an easily observable lay 
(the American Standard calls for visual 
recognition of lay, and measurements to be 
made across the lay), basic methods and 
definitions suffice; but when measurements 
are of surfaces having higher micro-finish, 
the surfaces are less apt to have a “definite 
lay,” requiring smaller samples, resulting 
in less representative measurement. 

Also, despite light following pressure, 
the stylus may damage the specimen. This 
risk increases the smaller the stylus radius 
(and the softer the piece). 

In the area where surface finish measure¬ 
ments are most needed, such as for gages 
and standards, present techniques and 
definitions become less and less meaningful. 

To conclude, the difficulty in measuring 
surface finish arises in being able to apply 
universal techniques and definitions to an 
almost indeterminate set of conditions. To 
distinguish between “flatness,” “wavi¬ 
ness,” and “roughness” for all grades of 
surface from rough-machined to highly 
polished is indeed difficult. Standard terms 
are particularly difficult to relate to the 
most accurate gages. The solution requires 
either a synthesis of methods or having 
more than one fixed set of techniques and 
terminology for surfaces of widely different 
quality. 

Elastic Deformation in the Gaging 
of Standards 

Large mechanical members, such as sur¬ 
face plates, masters, and machine mem¬ 
bers, have been shown to compress, bend 
and twist from gravity or from other ap¬ 
plied forces. 

Despite the extremely light gaging force 
associated with high-magnification me¬ 
chanical and electronic comparators, bend¬ 
ing, squashing, compression and bulging 
are just as much of a problem when gag¬ 
ing standards—especially in terms of the 
accuracies sought, Fig. 194. Loewen cites 
the example of a bar of quartz, 1 inch 
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fig. 195 -'-/?? most cases, compression of the 
gage, resulting from stylus ( indicator ) force, 
is negligible. Normally, it is discounted. 

figs. 196 , i 97, 198, 199, 200, 201—6 examples 
of how micro-geometry effects micro-inch 
[values of Yidh pm] measurement. Examples 


I S depict situations where overall 
force remains constant; local surface 
deformation, however, becomes progressively 
greater due to the shape of the objects, 
causing greater unit pressure. 

Example 6 illustrates an error caused by 
the probe falling alternately into the hills 


and valleys of the surface irregularities. 
This is potentially the biggest single source 

of error because it is random in nature. 
Moreover, the situations pictured in 
Examples 5 and 6 may be working in 
combination to increase error. 




[25.4 mm) in diameter and 10 inches 
[254 mm] long, resting on simple supports 
at either end; for every gram [0.035 oz.[ 
load at center, the center-point sags 1 
millionth of an inch [0.000025 mm].* 

Robert Hooke in 1676 published a gen¬ 
eral law as the result of experiments, show¬ 
ing that “the extension is proportional to 
the force. Hooke’s Law may be re-stated 
in terms of “Young’s Modulus,” or: 

Stress 

Modulus of Elasticity = ——— 

Strain 

“Young’s Modulus” of a material is the 
quotient obtained by dividing stress per 
square inch by elongation in one inch 
caused by this stress. 

FI 

The formula derived is: M= — 

ea 

where: M = Modulus of Elasticity 

F = Force applied 
1 = Length 
e = Elongation 
a = Area (cross section) 

Within their elastic limit, virtually all 
solid bodies are perfectly elastic. That is, 
when the force causing deformation is re¬ 
moved, the object returns to its normal 
shape. Hooke’s Law applies to either 
elongation or to compression. This is not 
to say that the “elastic limit” cannot be 
exceeded in the gaging process. In fact, it 
can accidentally be exceeded even with 

*Erwin G. Loewen, “Micro-inch Accuracy . . . 
Really?”, American Machinist , June 5, 1967. 


comparatively light gaging force because 
of surprisingly high unit pressures which 
sometimes accompany these measurements. 

It must be clearly pointed out here that 
this discussion is concerned only with 
forces that do not exceed the elastic limit 
(point of permanent deformation) of a 
specimen. 

“Young’s Modulus,” though derived 
empirically, is extremely useful in deriving 
the amount of deflection, given the material 
and the force applied. 

Heinrich Hertz, 1881, mathematically 
derived many of the formulas which are 
the basis of calculating deformations of 
various materials and shapes of bodies 
in contact. The classical Hertzian formula 
can be condensed into simple form* 

where: Y = deformation 
F = applied force 
D = diameter of contact 
K = a deformation variable de¬ 
pending on material, shape 
of bodies in contact, etc. 

Today’s closer tolerance requirements 
and the fact that parts are being made 
increasingly to dimensions rather than to 

*R. V. Anderson and A. L. Lauerman, “Contact 
Force (Measuring Pressure); Its Effect On Micro¬ 
inch Measurements,” Bulletin X7-995/070, North 
American Aviation, California. 


fit, has caused these formulas to be re¬ 
evaluated in terms of gaging. 

It is evident from the Hertzian formula 
that three elements influence deformation 
in measuring. (Compression from the mea¬ 
suring force, see Fig. 195, has been omitted 
from this analysis for simplicity, since it is 
normally a negligible amount). 

1. Measuring Force —Deformation in¬ 
creases with increasing measuring 
force— (to the % power of the force). 

2. Measuring Contact Area —With a 
total given force, the smaller the 
contact area on which this force is 
made to act, the greater the force 
per unit area (lbs./sq. in.) [kg/cm 2 ]; 
thus the greater deformation.* 

3. K— Deformation Variable —The “K” 
factor in actual practice may be com¬ 
posed of innumerable combinations 
of variables. For convenience they 
are classified below under two 
headings: 

a. Geometry of the Contact Area— Illus¬ 
trated in Figs. 196 through 199. 

b. Type of Material in Contact— Defor¬ 
mation becomes progressively less 
depending on whether the piece to be 
gaged is aluminum, steel, or carbide; 
similarly, if the probe is steel, car¬ 
bide, or diamond. 

*This mathematically demonstrates that when 
measuring surface finish, a needle type stylus probe 
may easily damage the specimen (exceed elastic 
limit) even under extremely light gaging force. 
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Providing certain assumptions are made, 
the most important of which is that the 
objects are perfectly smooth and homo¬ 
geneous, the amount of deformation or 
compression is predictable mathematically. 
In fact, controlled gaging experiments by 
Nickols and Oakley of the NPL demon¬ 
strated surprisingly close correlations with 
Hertzian values, the closer the approach 
to “pure conditions.”* 

Surface Finish and Deformation— From 
Fig. 200 and Fig. 201, it can be seen that 
two types of error arise as a result of the 
quality of the surface. 

In the first, and most serious instance, 
highly random errors arise from the probe 
falling alternately into the peaks and 
valleys of surface roughness. 

The second type of error is due to the 
uncertain amount of local deformation, 
increasing according to the roughness of 
the finish. 

Providing the exact same spot is gaged 
against, there should be little measured 
deviation, regardless of finish. In actual 
gaging practice, however, this is not always 
practical. A greater gaging force is recom¬ 
mended in proportion to the roughness 
of the finish, to partially flatten out the 
irregular pattern to some stable level. As 
a correlary to this, the lightest possible 
gaging force should be used where the 
very highest accuracy is desired of a sur¬ 
face finished to optical quality. 

Referring again to Figs. 196 through 
199, if Hertzian values were obtained from 
these examples, it would be found that 
when using only a few ounces [100 grams 
or so] of force, not only is deformation 
great, but moreover the differences in de¬ 
formation amount to many millionths of 
an inch [tenths of a ^m]. 

*L. W. Nickols and T. R. J. Oakley, “The In- 
fluence of Measuring Force, Stylus Radius and 
Surface Finish on the Accuracy of Measurement of 
Workpieces by a Comparator,” Proceedings of the 
Institution of Mechanical Engineers, London, 1961, 
pp. 195-207. 


To obtain the true correction values, 
complex calculations may be required. 
With today’s technology, computers may 
be used to derive many of these values, 
Fig. 202. 

From previous observations, two recom¬ 
mendations follow: 

1. Eliminate variables —All elements of 
comparison in the measuring set-up 
should be kept as uniform as possible. 
The comparison of unlike materials, 
shapes or types can cause unforeseen 
errors. If, for example, the specimen 
differs too far from the nominal size 
of the master, not only will the 
linearity of the comparator introduce 
an error, but the different position 
of the comparator needle will change 
the “force” applied. 

2. Apply a correction factor— If point 
No. 1 is not possible (seldom the 
case), either allow a larger margin 
of “uncertainty,” or make a careful 
analysis of the variables and apply 
the appropriate correction factor. 


It can be strikingly demonstrated just 
how careful an analysis should be: A probe 
of very small radius wears rapidly to a 
greater radius, in which case the theoretical 
deformation factor is completely changed. 
Worthen* mentions a less serious effect 
but nonetheless one which comes into 
consideration: a diamond probe, quite 
often used in millionths of an inch [tenth 
of a jum] measurements, is difficult to polish 
to a true radius because its degree of hard¬ 
ness varies in different axes. 

Type of Probe— The choice of probe or 
stylus-type is a subject that is much 
debated and little understood. Nickols and 
Oakley** show that even its construction 
is important; a composite type of probe 
where the ball is restrained in a holder is 
itself under stress. This condition may al¬ 
ter the theoretical deformation correction 
factor. 

*“Millionths Measurement Seminar” at Federal 
Products Co., November 19, 1968. 

**L. W. Nickols and T. R. J. Oakley, Idem. 



fig. 202 —Today’s technology enables the 
use of computers to more easily determine 
complex defetrmation values. 
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FIG. 203 —The difference in deformation 
occurring when a 34 inch [6.35 mm] steel 
ball is com,pared to a steel gage block using 
a 1'fjt.h inch \3.175 mm] diameter probe is 
calculated here. It is equal to approximately 
12 millionths of an inch [0.0003 mm\. 


4 oz. 



4 oz. 


^ in. diameter 


probe 
2500 in. ball 



.2500 in. gage block 
d. 


Total deformation 
(di + d 2 ) 


Total deformation 
(d 3 + d 4 ) 


24.7 millionths inch 12.8 millionths inch 

Difference in deformation: 

24.7— 12.8 = 11.9 millionths in. 


All materials of steel 



20.2 millionths inch 0 millionths inch 


(Negligible) 


Difference in deformation: 
20.2 — 0 = 20.2 millionths inch 


All materials of steel 


Actually, no hard and fast rules can be 
applied as to the choice of probes. Each 
measurement problem requires separate 
analysis, as shall be shown. 

Some metrologists, realizing that the 
amount of deformation is least when a flat 
anvil-type probe is used, assert that it should 
be used wherever possible. The value of a 
flat, anvil-type probe for most types of com¬ 
parator measurement is of questionable 
value. The parallelism of probe to the oppos¬ 
ing support surface is perpetually suspect. 
Errors in measurement may far surpass 
those due to “deformation.” 

As a rule, where objects having plane sur¬ 
faces are compared, a large radius probe is 
preferred over one having a small radius for 
the following reasons: 

1. The danger of brinelling is lessened. 

2. Wear of the probe is minimized. 

3. Deformation is less, requiring smaller 
corrections. 

This does not necessarily mean that a 
large radius probe is preferred over a small 
radius under all circumstances. One obvious 
limitation is the internal measurement of 
small radii. It has been our experience that 
for general shop use of electronic indicators, 
where conditions are less than perfect, a 
large radius probe is inclined to collect min¬ 
ute dust particles, or to have slight oil films 
adhere; these would normally be “wiped 
off” with a small radius probe. 

Nor should it be assumed that minimal 
deformation is to be sought at the expense 
of all other considerations. This can be 
proven in purely Hertzian terms and simul¬ 
taneously give some idea of the magnitude 
of errors which occur through elastic defor¬ 
mation using a comparator.* 

Case No. 1, Fig. 203—a 34 in- steel ball is 
gaged for size by comparison to a .250 in. 
steel gage block, using a probe-tip of 34 in. 


*For further information on the deformation of 
various other geometrical shapes when in contact, 
see S. Timoshenko and J. N. Goodier, Theory of 
Elasticity, New York, 1951. 


diameter and 4 oz. applied measuring 
force [3.17 mm, 113.4 grams]. 

Referring to Fig. 203 again, deformation 
from applied measuring force on spherical 
contacts at di: 


di = 1.23 


,//PY 

Rl + R -2 

J W 

__ R4R2 _ 


which calculates to be 14.6 millionths inch. 

Deformation from applied measuring 
force of ball to its support at d 2 : 


d 2 = 1.23 



which calculates to be 10.1 millionths inch. 

Total deformation, di+d 2 = 14.6+10.1 
or 24.7 millionths inch. 

Now consider the deformation from ap¬ 
plied measuring force of a 34 inch diameter 
probe against the gage block, Fig. 203. 


Deformation at d 2 : 

d - 1 ' 23 yj (l)' x H 

calculated to be 12.8 millionths inch. 

If we assume that the deformation of the 
gage block to its support (d 4 ) is negligible, 
then total deformation equals 12.8+0, or 
12.8 millionths inch. 

This means that if no correction factor 
were applied to allow for deformation, the 
error in the calibration of the 34 inch steel 
ball would amount to 24.7 — 12.8, or 11.9 
millionths inch.* 

Case No. 2 —A 34 inch steel ball is gaged 
for size by comparison to a .250 inch steel 
gage block, using a flat anvil probe, 4 oz. 
applied measuring force. 

Referring to Fig. 204, deformation of ball 
to its support, d 5 , as at d 2 is 10.1 millionths 
inch. 

Since deformation at d6 equals deforma¬ 
tion at d 5 , total deformation is 10.1 + 10.1, 
or 20.2 millionths inch. 

Again, deformation both at d 7 and d 8 is 
negligible, or 0 millionths inch. 

*See Figs. 203 & 204 for metric conclusions. 


FIG. 204 —The difference in deformation 
which occurs when the steel ball shown in 
Fii]. 203 is compared to the same gage 
block using a flat probe is approximately 
20 millionths of an inch [% pm]— 
almost twice as great as that when using a 
Vg-inch [3.175 mm] diameter probe. 






FIG. 205 —The roller-cheeking device uses a 
%-inch [3.175 mm] radius diamond probe, 
and is able to detect, differences of one or 
tu !0 millionths of an inch [‘/aom of a pm 
or less]. Rollers are used in many 
precision mechanisms, such as in spindles 
and roller-ways. 


This means that if no correction factor 
were applied to allow for deformation, the 
error in the calibration of thesteelball would 
amount to 20.2—0, or 20.2 millionths inch. 

Comparing Case No. 1 to Case No. 2, the 
difference in deformation is almost twice as 
much when using a flat probe as opposed to a 


in. radius probe. 


Using 1 lb. load: 

Deformation 

l /% in. dia. probe 


against 1 in. 
dia. ball. 

. 26 millionths in. 

Vs in. dia. probe 


against 1 in. 
gage block .... 

. 25 millionths in. 

1 in. dia. probe 


against 1 in. 
dia. ball. 

. 15.5 millionths in. 

1 in. dia. probe 


against 1 in. 
gage block .... 

. 12.5 millionths in. 


In this latter case, the large radius probe 
would call for a 3 millionths in. correction, 
whereas the in. probe would call for 
only a 1 millionths in. correction. 

The author realizes that the above calcu¬ 
lations need the clear light of experimenta¬ 
tion to be truly meaningful; however, it is 
only intended to point out that; 

1. Deformation values can be very great. 

2. It is not always “obvious” what type 
of probe is preferred. 

Deformation in itself is not necessarily 
deleterious to accuracy, so long as it is con¬ 
stant. An example is the roller-checking de¬ 
vice, Fig. 205, diagrammatically shown in 
Fig. 206. An optically-polished diamond 
probe of in. radius is used. 

Although there is undoubtedly more de¬ 
formation than if a fiat probe were to be 
used, actual size is secondary in importance 
to holding all rollers alike in size. 

Where actual size is to be determined, a 
master plug gage is preferred over a gage 
block.* 

*See Figs. 203 & 204 for metric conclusions. 




FIG. 206 —A schematic of the roller-checking 
device in Fig. 202. Although very simple 
and practical, it is believed this is the firs! 
design to employ the principle of gage 
blocks for an anvil and a “backstop.” 
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FIG. 207— NPL-Hilger Inter Jerome ter 
Courtesy of Rank Precision industries, Ltd, 
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“The optical system is illustrated here. The light from one 
of two discharge tubes A is focused by a condenser lens B 
onto the entrance slit C which is set at the focus of an achro¬ 
matic objective D. The light from the objective emerges as a 
parallel beam, is dispersed by the constant-deviation prism E, 
and finally reflected down to an optical flat F, which is sup¬ 
ported above, and at a slight angle to, the upper surface of the 
gauge G. The lower surface of the optical flat is coated with a 
layer of bismuth oxide to allow for transmission and reflection 
of the light. The light then passes to the upper surfaces of the 
gauge and the steel platen H, to which the gauge is wrung. The 
rays from these two surfaces are reflected back in the reverse 
direction along a slightly different path so that they emerge 
from an exit slit above the entrance slit. A mirror J is situated 
behind the plate I to reflect the light into the telescope. 


The optical flat can be moved up or down depending on the 
length of the gauge to be measured; this adjustment is neces¬ 
sary since brighter and sharper fringes are obtained with a 
reduction in path length. By inclining the optical flat, two sets 
of parallel interference fringes will be seen: one set from the 
face of the gauge, and the other from the surface of the platen. 
The fringes will be displaced, however, and in order to measure 
the length of a gauge it is necessary to determine the fringe 
fraction for three or four wavelengths—that is, the fringe dis¬ 
placement in terms of the fringe spacing. 

If a is the displacement, b the fringe spacing, the fringe frac¬ 
tion f = alb (above). This fraction can be estimated to 
of a fringe spacing or better. An adjustable cross-wire C may 
be used.” 


Interferometric Measurement of Length 
Gage blocks and other end standards may 
be calibrated to a high resolution of accu¬ 
racy by interferometric means. Fig. 207 
shows the NPL-Hilger Gauge Interferom¬ 
eter; Fig. 208 gives a diagrammatic expla¬ 
nation of the system. Fig. 209 shows the 
Tsugami Gage Interferometer; Fig. 210, its 
optical arrangement. 

When measuring by “absolute” means, 
the case with the two systems illustrated, it 
is necessary to determine which particular 
half wavelength is registering, since only 
the fractional value of a fringe is evident 
to the observer. The possibility exists that 
there may be many multiples of X/2 de¬ 
viation. 

This calculation can be accomplished 
fairly easily by one of two methods: 

In the first, the gage block can be com¬ 
pared to another gage block whose length 
is known to less than one fringe. However, 
it is more common for the gage to be 
calibrated by 2 or 3 light sources of 
varying wavelengths; by a process of “co¬ 
incidence”, then, the correct wavelength 
is calculated. The manufacturer often fur¬ 
nishes a special slide-rule to facilitate this 
calculation. 

Total Phase Correction —Just as mechani¬ 
cal measurements entail “corrections,” the 
same is true for optical measurements. 
“Phase correction” refers to a correction 
which must be made due to a “phase shift” 
of light waves when the incident ray is 
reflected from materials other than opti¬ 
cally finished glass or quartz (which have 
“zero” phase shift). In perfectly lapped 
steel, the phase change correction amounts 
to approximately .0000008 (.8 millionths) 
inch [0.00002 mm]. 

Total phase change correction generally 
includes the total amount that the inci¬ 
dent ray of light appears to reflect from 
below what would be called the mechanical 
surface of the object, sometimes referred to 
as “penetration”, Fig. 211. 


fig. 208— Optical system of the 
NPL-Hilger Interferometer 
Courtesy of Rank Precision Industries, Ltd, 
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FIG. 209 —Tsugami Gage Interferometer. 

Courtesy of Tsugsrni Mfg, Co., Ltd. 





LENGTH MEASUREMENT OF SLOCK GAGE 

Interference fringes formed on sur¬ 
face of block gage (inside patterns) 
and baseplate (outside patterns). 



Interference fringes formed on sur¬ 
face of baseplate by light passing 
through vacuum (inside patterns) 
and air (outside patterns). 


FIG. 210 — Optical arrangement of the 
Tsugami Gage 1 nterferorneter. 

Courtesy of Tsugami Mfg. Co.. Ltd. 









Reflected Incident 

light light 


Mechanical surface 



The correction factor varies from one 
material to another. It also varies with the 
surface finish. In the case of gage blocks, the 
amount of phase change varies from manu¬ 
facturer to manufacturer, or even for the 
gage blocks within a particular set. Total 
phase change correction may amount to as 
much as 2 or 3 millionths of an inch [0.00005 
to 0.00007 mm] for gage blocks, depending 
on surface finish. 

Wringing Film Thickness— By definition, 
the length of a gage block includes the thick¬ 
ness of one wringing film. The thickness of a 
wringing film is approximately millionth 
of an inch [0.000006 mm] regardless of the 
lubricant selected. However, an improper 
wring may easily result in a value many 
times that amount. 

Silicone or filtered kerosene are two lubri¬ 
cants suggested, the latter perhaps the more 
preferable. It is sometimes specified that 
surfaces which are to be wrung together 
should be absolutely dry. While admit¬ 
tedly the lubricant should be wiped ever- 
so-thin, there is as much danger from too 
little lubricant as from too much. A truly 
clean surface, though difficult to attain, 
may not wring satisfactorily. 

The necessity of being able to measure 
these thin films adds to the difficulty of 
interferometric measurements. Adding fur¬ 
ther to the complexity of this measurement 
problem is the difficulty of separating the 
effects of wringing film and phase “pene¬ 
tration,” and at the same time measure each 
exactly. The refractive index of the film 
must also be known. There is even the possi¬ 
bility that the thickness of the film depends 
on the type of materials in contact, their 
flatness and surface finish. 

Since light wave emission (krypton 86) is 
now the official standard of length, and since 
the surfaces of length standards are used as 
reflectors in the optical set-up, means are 
being sought to measure these values as ex¬ 
actly as possible. Young reports that “the 
uncertainties inherent in these methods 


presently limit the measurement of length 
to ±1 micro-inch [±/4oth /«n].* 

The above considerations are of greatest 
concern in measurements below the micro¬ 
inch range. P. Kogurt states that from his 
practical experience, and the author is in¬ 
clined to share his view, the most important 
factor in obtaining reliability in the mea¬ 
surement of a gage block is simply that it 
must have good geometry.** 

3. Measurement in Science 
and Industry 

The history of “official” length standards 
has been traced separately from the main¬ 
stream of the history of measurement for 
purposes of continuity. Standing alone, such 
a chronology is somewhat misleading and 
historically inaccurate. The evolution of 
measurement in all of its forms of which 
length is only one aspect cannot be com¬ 
prehensively presented except when related 
to concurrent scientific and industrial de¬ 
velopment. 

The history that follows pursues three 
main currents which will be seen to be con¬ 
tinually cross-influencing: measurement,ma¬ 
chine tools, and interchangeability. 

Interchangeability is contingent on mea¬ 
surement. This fact should be readily ap¬ 
parent, for through interchangeability, 
parts can be made to match and to assemble 
without trial-and-error fitting, providing 
they are first made to exact dimensions. The 
relationship of machine tools to measure¬ 
ment, however, is less obvious. 


*T. R. Young, “Analysis of Phase Correction In 
Interferometric Length Measurement,” NBS Re¬ 
port No. 9022, September 30, 1965. 

**Author’s conversation with P. Kogurt, Chief 
Metrologist of the Eli Whitney Metrology Labora¬ 
tory, Sheffield Corporation, May 21, 1969. 


FIG. 211 —. When measuring standards by 

interferometric means, a cerrectien must be 
applied to relate optical t» mechanical 
length. 


Incident light undergoes a phase change 
and appears to “penetrate” the surface, 
reflecting from a point below what might be 
called its mechanical surface. “Penetration” 
als$ varies with surface finish. 
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SIGNIFICANCE OF 
MACHINE TOOLS 

Man is a tool-making animal. His tools are 
an extension of his limbs and his intellect; 
they enable him to perform tasks faster and 
more proficiently than his body alone is 
capable of doing. The entire physical level 
of civilization depends on the level of sophis¬ 
tication attained in making and using tools. 

Machine tools are man’s most significant 
tools, since their purpose is to create other 
tools, instruments and machinery. To make 
a distinction, machinery such as a weaving 
machine or a printing press may solve one 
problem, but have no heirs. A machine tool 
such as a lathe or jig borer, on the other 
hand, solves many problems in addition to 
generating a vast progeny. Indeed, one def¬ 
inition of a machine tool is that in the ma¬ 
chinery family, only the machine tool can 
reproduce itself.* 

Our present high standard of living is de¬ 
rived from mass-production. Mass-produc¬ 
tion is not possible without interchangeabil¬ 
ity. The level of interchangeability depends 
on accuracy of tools. The accuracy of a tool, 
such as a stamping die, depends ultimately 
on the machine tools which made it. Be¬ 
cause the accuracy of machine tools is the 
essential consideration, it is revealing to 
trace the development of some of the more 
accurate machine tools and to note their 
historical contribution. 

One notable characteristic of outstanding 
mechanics and machine tool builders would 
seem to be their obsession with accuracy. 
The significance of this relationship is an 
important part of the following chronicle. 

The Industrial Revolution, conceded to 
have begun in England about 1710 with 
Newcomen’s atmospheric steam engine, and 
to have reached full thrust by 1800, is one 
of the most important stages in man’s his¬ 
tory. It is at this time that man truly began 

*Fred H. Colvin, “60 Years with Men and Ma¬ 
chines—An Autobiography”, p. 6. 


to multiply his efforts through the use of 
machines. A major but too-little explored 
impetus behind the dynamic growth attend¬ 
ing this period is the development of ma¬ 
chine tools from 1790 to 1840, first in Eng¬ 
land and later in America. In point of fact, 
the designs of basic machine tools today are 
fundamentally the same as they were in the 
middle of the nineteenth century. 

WHAT DELAYED 
THE DEVELOPMENT 
OF MACHINE TOOLS? 

An Englishman named Maudslay, more 
than any other individual, deserves credit 
for the accelerated development of machine 
tools. This mechanic and machine tool 
builder was then at the peak of his inven¬ 
tiveness (seepage 142). However, why did 
this development begin in England, and 
why not until the time of Maudslay? 

The essential ingredients had long been in 
existence. To begin with, there were many 
skilledcraftsmen. Early mechanical devices, 
such as a clock installed in Salisbury in 
1386, showed a high degree of craftsman¬ 
ship. These devices were built by skilled 
blacksmiths; they were not the products of 
machine tools. 

Secondly, basic mechanical principles 
were known. The pole lathe, Fig. 212, for 
example, dates to antiquity. The screw 
thread, one of the most fundamental prin¬ 
ciples of a machine tool, traces to Archi¬ 
medes, who introduced it as a means of 
raising water. An old illustration of 1483, 
Fig. 213, shows that mechanical means were 
used to produce screw threads out of wood 
with a true compound slide rest 300 years 
before the time of Maudslay, with whom 
this development is usually associated.* 

Thirdly, there was no want of inventive¬ 
ness. Renaissance man was fully as inven- 


*Edwin A. Battison, “Screw-Thread Cutting by 
the Master-Screw Method Since 1480,” United 
States National Museum Bulletin 240, Smithsonian 
Institution, Washington, D.C., 1964. 


fig. 212 — The pole lathe, circa 1800, dates 

to antiquity. It employs principles common 
to the modern lathe. 

Lent to Science Museum by Thomas Noaks and Sons, 
Ltd., London, 



FIG. 213 — Screw-cutting lathe dating to 
about lf.83 produced screw-threads out of 
wood and employed a true compound 
slide rest. 

Courtesy Hausbuch, Germany. 
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fig. 214— Da Vinci sketched a boring mill 
with a self-centering chuck about 1500. 
Courtesy of Science Museum, London. 



tive as his counterpart of the Industrial 
Revolution. The world may never again see 
a DaVinci, who in 1500 anticipated in his 
sketches machine tool principles which 
would not evolve for nearly 300 years. Such 
devices as a boring mill with a self-center¬ 
ing chuck, Fig. 214, an internal grinding 
machine, and a screw-cutting lathe with 
change-wheels to obtain different leads Fig. 
215 are among his many drawing-board in¬ 
ventions. 

Finally, there was no want of scientific 
method. E arly instrument and clock-makers 
in France, such as Jacques Besson (1569), 
who was known especially for ornamental 
turning, were making items requiring highly 
developed skills. Charles Plumier, more¬ 
over, published L’art du Tourneur as early 
as 1701. 

Renaissance man, it should be noted, did 
not put into practice the products of his 
thinking. His complex mechanisms were 
rarely built. Consequently, no lineage or de¬ 
velopment ensued. It was not until machine 
tools were fashioned from metal and de¬ 
signed, in addition, to cut metal that ma¬ 
chine tools came into their own. While the 
instrument-makers had precision equip¬ 
ment, often made of metal, their skills did 
not extend to machine tools. The machine 
tools they did use remained primitive, as a 
result. 

The Chinese and Indians were familiar 
with the techniques of producing crucible 
steel before the time of Christ. Europe used 
crude methods of carbonizing steel until the 
process of producing crucible steel was re¬ 
discovered by Benj amin Huntsman in 1746. 

Sweden had vast timber resources and a 
good supply of high grade iron; during one 
period she supplied England with as much 
as four-fifths of its wrought iron. Although 
definite proof is lacking, some records, such 
as those from the Swede, Polhem (1740), 
describe the use of heavy duty industrial 
lathes for turning rollers to roll strip and 
bar iron; they even trued by grinding. Swed- 


fig. 215 — Renaissance man designed but 
often did not build his mechanical 
contrivances; note this screw-cutting lathe 


with change wheels to obtain different leads, 
modeled after sketch by Da Vinci. 

Courtesy of Science Museum, London. 

British Crown Copy tight. 
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ish influence was not spread, however. To 
be sure, the Swedes were great travelers, but 
they concerned themselves with looking for 
ideas, not spreading them. 

Old records and prints indicate that many 
of the first machine tools, dating to the 
early eighteenth century, were invented by 
the French. A good example is Vaucanson’s 
lathe of 1765, designed along modern prin¬ 
ciples using two inch [38.1 mm] square 
bars set on edge as prismatic guideways 
(an early double-V). While ingenious, these 
inventions rarely found application. 

It is in England that machine tools were 
first given finished form and excellence. The 
first machine tools offered for sale were con¬ 
structed in Matthew Murray’s famous 
Round Foundry about 1800. Indeed, well 
into the nineteenth century, both France 
and Germany imported most of their ma¬ 
chine tools from England. 


EVOLUTION OF CANNON¬ 
BORING MACHINES 

A sequence of events following the evolu¬ 
tion of cannon-boring machines and the 
consequent development of the boring mill 
can be credited with sparking the Industrial 
Revolution. They also explain why the 
forces of change took hold in England and 
then spread throughout the rest of Europe. 

The art of boring cannons goes back at 
least three or four hundred years. An old 
print of 1540 shows a water-powered can- 
non-boring mill. The earliest methods were 
not accurate primarily because the boring 
tool was guided by the cast hole. Maritz, a 
Swiss, invented a mill to allow boring from 
solid, probably about 1713. He went to the 
Netherlands Gun Foundry at The Hague, 
where the method was adopted. The tech¬ 
nique was further improved in 1755 by 


Verbruggen, Fig. 216. After his discharge 
from The Hague Foundry in 1770, Verbrug¬ 
gen was appointed Master Founder at the 
Royal Arsenal of Woolwich, England. 

By the early 1700’s, England had virtu¬ 
ally depleted its supply of wood. In 1711, 
Abraham Darby began to mine coal for 
the smelting of iron. Increased use of coal 
gave Newcomen the idea of developing a 
workable steam engine for mine-pumping. 
Newcomen’s design was that of an “atmos¬ 
pheric” engine. Steam lifted a piston, and 
the pumping was accomplished on its fall, 
facilitated by a vacuum created by cooled, 
condensed steam. Most of the power from 
the steam, however, was wasted. 



FIG. 216 . A gun-boring machine by 

Verbruggen ( 18th Century) was an 
important step in the evolution 

of boring machines. 

Courtesy of Science Museum, London. 

British Crown Copyright. 
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James Watt 

James Watt’s steam engine had great his¬ 
torical impact, but its success had to await 
an improvement provided by machine tool 
technology. The design of Watt’s steam en¬ 
gine differed significantly from New¬ 
comen’s. The piston stroke was powered by 
steam which required “that the cylinder be 
kept always as hot as the steam that en¬ 
tered it.”* It was the first true steam engine 
as we know it today. 

Watt was confronted with several prob¬ 
lems in building his engine, but because of 
his early training as an instrument maker, 
he was always guided by scientific princi¬ 
ples. Such refinements as the “condenser,” 
for example, resulted from Watt’s knowl¬ 
edge of the nature of steam. 

The prime reasons Watt’s engine was so 
long in the making is that, first of all, the 
finer workmanship required for its design 
was unavailable. Part of this problem was 
alleviated, however, by his alliance with a 
mechanic and foundryman, Matthew Boul¬ 
ton. Secondly, the success of his engine was 
contingent on establishing a true bore in 
his cylinder. Carnegie states the problem: 
“It was still the cylinder and its piston that 
gave Watt the chief trouble. No wonder the 
cylinder leaked. It had to be hammered into 
something like true lines, for at that day so 
backward was the art that not even the 
whole collective mechanical skill of cylin¬ 
der-making could furnish a bored cylinder 
of the simplest kind.”** 

Wilkinson’s Cannon-Boring Machine 

Watt’s greatest difficulty was finally solved 
in 1776 by John Wilkinson, an iron-master. 
Wilkinson’s cannon-boring machine, used 
to bore Watt’s steam-engine cylinder, was 
much like that of Verbruggen, with which 
Wilkinson was entirely familiar. The boring 
bar was guided from both ends, independent 
of the casting, Fig. 217. 

*Andrew Carnegie, James Watt, p. 41. 

**Ibid., p. 46. 


His main obstacle overcome, Watt was 
elated: “Mr. Wilkinson has improved the 
art of boring cylinders,” he exclaimed, “so 
that I promise you a 72-inch [1828.8 mm] 
cylinder being not further from absolute 
truth than the thickness of a thin sixpence 
in the worst part.”* Commenting on Wil¬ 
kinson’s invention, J. Robert Moore wrote: 

“Wilkinson’s contribution, in the light of 
present-day knowledge, appears exceed¬ 
ingly simple. Actually, it was a tremen¬ 
dous forward stride. It embodied, ap¬ 
parently for the first time, the idea of 
generating an internal cylindrical surface 
by boring around a fixed axis. This simple 
invention was an enormous contribution 
to mechanical progress. Without it, a host 
of present-day devices, especially those 
embodying hole location would be impos¬ 
sible to mass-produce.”** 

It is doubtful that Wilkinson was the first 
to generate a hole in this manner. More 
likely, the reason for this assumption is its 
association with boring the cylinder of the 
first truly efficient steam engine. In any 
case, this concept influenced the develop¬ 
ment of hole-location devices, and was an 
early ancestor of the jig borer. 

Steam engines became the preoccupation 
of nearly every mechanic in England. 
Engineers such as James Nasmyth “cut 
their mechanical teeth” on this fascinating 
development. Steam engines accelerated the 
industrial growth of England. Power could 
now be brought to an industrial location in¬ 
stead of having to depend on proximity to 
water, as was the case in America. This was 
the reason for Boulton’s initial interest in 
the steam engine. The shift to steam¬ 
generated energy prompted many techno¬ 
logical advancements. The development of 
“self-acting” machine tools, for example, 
was a major product of steam technology. 

*L. T. C. Rolt, Tools for the Job, p. 56. 

**J. Robert Moore, Precision Hole Location, p. 55. 







FIG. 21S— Maudslay’s original screw-cutting 
lathe, circa 1800. 

Courtesy of Science Museum, London. 



MACHINE TOOL 
DEVELOPMENT IN ENGLAND 

Henry Maudslay (1771-1831) 

In the significant period between 1790 and 
1840, one man—Henry Maudslay—ranks 
above all others in mechanical skill and 
contributions to machine design. He was 
born in Woolwich, August 22, 1771 and, at 
an early age, learned many machine princi¬ 
ples at Woolwich Arsenal. He is associated 
with the development of the lathe slide rest, 
as well as other machine tool innovations. 
Roe calls the slide rest “one of the great 
inventions of history!”* 

The reasons for Maudslay’s acknowl¬ 
edged fame as a mechanic have not always 
been understood. His prismatic way design 
in his first classic lathe was pre-dated by 
Vaucanson; the design of his screw-cutting 
lathe, Fig. 218, was strongly influenced by 
the work of Jesse Ramsden (1735-1800), an 
instrument-maker whose screws were used 
to make precision linear scales; Maudslay 
was never identified, as was Murray, with 
the development of machine tools for sale. 

Maudslay’s importance was actually two¬ 
fold. The essential characteristics of a ma¬ 
chine tool builder came together in one man 
for the first time: He understood the con¬ 
cept of generously using iron, steel and 
brass, as opposed to wood, for making metal 
parts, and he brought to bear the skill and 
good design sense of the instrument maker. 
Secondly, he must be appreciated as an 
historical figure who greatly influenced 
subsequentevents. The legacy of Maudslay, 
the machine tool builder, can be measured 
by the eminence of his disciples—Bramah, 
Clement, Whitworth, Nasmyth—all of 
whom either worked in his shop or came 
under his influence, Fig. 219. 

An insight into the character of Maudslay 
comes from Nasmyth’s notations of some of 
his favorite maxims: 

*Joseph W. Roe, English and American Tool 
Builders, p. 36. 


fig. 219 —-James Nasmyth, like many 
famed mechanics, was strongly influenced 
by Maudslay’s principles. Shown 
is Nasmyth’s original drawing of his 
invention—the steam hammer. 

Courtesy of Science Museum, London. 
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FIG, 220 —Believed to be, Maud slay’s “Lord 
Chancellor ,”t a measuring instrument 
which employed a master screw having 100 
threads l* the inch [approximately f threads 
per millimeter}. H could deled minute 
differences in length, circa 1800. 

Courtesy of Science Museum, London. 



“First, get a clear notion of what you de¬ 
sire to accomplish, and then in all proba¬ 
bility you will succeed in doing it.” 

“Keep a sharp look-out upon your mate¬ 
rials; get rid of every pound of material 
you can do without; put to yourself the 
question, ‘What business has it to be 
there?’ Avoid complexities, and make 
everything as simple as possible.” 

“Remember the get-at-ability of parts.”* 

These homespun remarks embody the 
very essence of good machine tool design. 

Perhaps the most important factor con¬ 
tributing to Maudslay’s success, however, 
was his devotion to accuracy, notably in 
the creation of true planes. Maudslay was 
also fascinated with the principle and use of 
the accurate screw-thread. Nasmyth ac¬ 
knowledges that the first to use a screw as 
a measuring element was William Gascoigne 
(1648). Yet he goes on to say: “The produc¬ 
tion of perfect screw threads was one of 
Maudslay’s highest ambitions and his prin¬ 
cipal scientific achievement.”** 
Maudslay’s “LordChancellor”, Fig. 220, 
a measuring instrument having 100 threads 
per inch [approximately 4 threads per mm], 
was recently found to be amazingly accu¬ 
rate when tested by modern methods.*** 
Nasmyth describes the instrument: 

“Not only absolute measure would be ob¬ 
tained by this means, but also the amount 
of minute differences could be ascertained 
with a degree of exactness that went quite 
beyond all the requirements of engineer¬ 
ing mechanism; such, for instance, as the 
thousandth part of an inch [0.0254 mm]! 
It might also have been divided so far 
as a millionth part of an inch [0.000025 
mm] but these infinitesimal fractions 

*James Nasmyth, Autobiography, p. 127. 

**W. A. S. Benson, “The Early Machine Tools 
of Henry Maudslay,” Engineering, February 1, 
1901, p. 134. 

***C. S. Davison, “Report on Maudslay’s Mi¬ 
crometer—1805,” Science Museum, London, Sep¬ 
tember 23, 1959. 


have really nothing to do with the effec¬ 
tive machinery that comes forth from 
the workshops, and merely show the 
mastery we possess over materials and 
mechanical forms.”* 

While Maudslay was undoubtedly influ¬ 
enced by the works of others, he was none¬ 
theless an originator of ideas. 

Joseph Whitworth (1803-1887) 

Perhaps better known than Maudslay, 
Joseph Whitworth, born in Stockport, 
Cheshire in 1803, deserves credit more for a 
high standard of accuracy, than for true in- 

Wnpublished letter to author from Robin Gos¬ 
ling, Science Museum, London, August 26, 1966 
indicates that researchers are assured this instru¬ 
ment is unquestionably Maudslay’s “Lord Chan¬ 
cellor”. 

Names Nasmyth, Ibid., p. 146. 


spiration. The ideas he espoused were pri¬ 
marily elaborations of principles first en¬ 
countered while working in Maudslay’s 
shop. 

In a Presidential Address to the Institu¬ 
tion of Mechanical Engineers in 1856, Whit¬ 
worth spoke of . . the vast importance of- 
attending to the two great elements in con¬ 
structive mechanics, namely, a true plane 
and power of measurement. The latter can¬ 
not be obtained without the former, which 

is, therefore, of primary importance ... all 
excellence in workmanship depends upon 

it. ”* 

The classical method of creating a flat 
plane by the three-plate method is usually 
credited to Whitworth, although Nasmyth, 

* Joseph W. Roe, English and American Tool 
Builders, p. 99. 
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fig. 221 — The three-plate method of 
generating a flat plane traces tu Maud-slay. 
Whitworth probably first employed 
hand-scraping to generate flatness, as in 
these scraped plates which he prepared. 
Courtesy of Science Museum, London, 

British Crown Copyright. 



who apprenticed under Maudslay, attrib¬ 
utes this development to his mentor 
Maudslay. It is likely that Whitworth sub¬ 
stituted hand-scraping for “grinding” (lap¬ 
ping with abrasive powder) in the final 
finishing, and that he first did this while in 
Maudslay’s shop, Fig. 221. 

Whitworth exhibited in 1855 a “Measur¬ 
ing Machine,” Fig. 222, based on “end mea¬ 
surement,” which, it is said, could detect 
differences as small as one-millionth of an 
inch [0.000025 mm]. In all probability, this 
machine did not originate sizes, but simply 
compared minute differences. 

Whitworth said this about the advan¬ 
tages of end measure: 

“We have in this mode of measurement all 
the accuracy we can desire; and we find 
in practice in the workshop that it is easier 
to work to the ten-thousandths of an inch 
[0.0025 mm] from standards of end mea¬ 
surements, than to one-hundredth of an 
inch [0.25 mm] from lines on a two-foot 
rule [610 mm]. In all cases of fitting, end 
measure of length should be used, instead 
of lines.”* 

The Whitworth Works rapidly grew in 
prestige, and by 1860 was the best-known 
throughout England. Its reputation began 
to wane after this date, however, largely be¬ 
cause of Whitworth’s autocratic approach 
to business. Nevertheless, his contribution 
was enormous and is fittingly commemo¬ 
rated in the “Whitworth Room”, Fig. 223, 
of the Institution of Mechanical Engineers 
in the United Kingdom. 

MEASUREMENT 

IN AMERICAN INDUSTRY- 

INTERCHANGEABILITY 

While England was to achieve early promi¬ 
nence in machine tools and steam engines, 
quite another chain of circumstances was 
occurring in America, distinctive to Ameri- 

* Joseph W. Roe, English and American Tool 
Builders, p. 101. 


FIG. 222 — Whitworth's “Millionth of an 
inch” l 1 /* pm] measuring machine of 1855. 
Lent to Science Museum ’•y Sir W. G. Armstrong, 
Whitworth & Co., Ltd., London. 
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i ic. 7 lib “Whitworth R»om’’ 
commemorates the enormous contributions 
of Joseph- Whitworth to mechanics. 

Courtesy of Vickers. Ltd. 
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can conditions. These circumstances ulti¬ 
mately led to the concept of interchange- 
ability. 

America’s adoption of the system of inter¬ 
changeability quickly gave it world promi¬ 
nence and provided a standard of living the 
rest of the world never dreamed possible. 
Europe was not aware of this system until 
after 1850, by which time it had been so 
firmly established in America that it was 
designated “The American System.” 

Whilehistorians may disagree, the author 
is convinced that the adoption of this con¬ 
cept had more to do with America’s rapid 
rise to world prominence than any other 
single factor. 

The enthusiasm with which the inter¬ 
changeable method of manufacture was 
embraced in America encouraged the 
development of machine tools. America’s 
machine tool development, while not as 
early, was rapid, nearly paralleling that of 
England. 

The idea of making parts interchangeable 
was first projected by armorers in France in 
1717. Jefferson, while ambassador to 
France, saw this approach attempted by 
the gunsmith, Le Blanc, in 1785. With his 
mechanical bent, Jefferson was aware of 
the benefits to be had from this system, 
but he failed to persuade Le Blanc, through 
official channels, to emigrate to America. 
Interchangeability was also unsuccess¬ 
fully attempted by Bodmer of France 
in 1811. 

Eli Whitney 

Eli Whitney . . came from that best 
school of mechanics, the N ew England hill 
farm.”* He is justly credited with the first 
successful application of interchangeable 
manufacture. 


*Joseph W. Roe, English and American Tool 
BuilderSy p. 146. 


Whitney wasfarfrombeing solely a farm- 
boy, however. Yale-educated, he had mo¬ 
bility in select circles, and utilized his 
associations to secure contracts from the 
government for manufacturing rifles. After 
years of wearisome, frustrating and largely 
unsuccessful negotiations to secure remu¬ 
neration from Southern cotton-growers for 
patent-infringement on his cotton gin, he 
turned his energies elsewhere. 

In 1798, he was awarded a contract by 
the government to manufacture 15,000 
muskets on an interchangeable basis. The 
government had two reasons for acceding to 
so unheard of a proposition: the first can 
be attributed to Jefferson’s influence; he 
understood the principle, and encouraged 
Whitney through the difficult years. The 
second was the government’s awareness of 
their dependence on French supplied mus¬ 
kets in the recent war against the British. 
Now that France herself was a threat to the 
new nation, the government could not dis¬ 
miss the prospect of receiving so many 
needed muskets in the short delivery time 
promised by Whitney. 

Whitney chose to set up his factory in 
New Haven, Connecticut. He deliberately 
employed essentially untrained workmen 
who had no preconceived notions about his 
system to “unlearn”. With a minimum of 
resources, his undertaking called for heroic 
effort. Whitney built a dam for power, 
erected a factory, trained men and designed 
his own gages and machine tools. 

This undertaking consumed him to a 
point where he worked endless days. As an 
apparent consequence, he did not marry 
until late in life. So immense was the task 
that completion of the musket contract was 
delayed many years; for unlike the me¬ 
chanics in England, Whitney had to work 
out his principles alone. 

In 1801, Jefferson wrote M onroe of Whit¬ 
ney: “He has invented molds and machines 
for making of his locks so exactly equal that 
take 100 locks to pieces and mingle their 


parts, and the 100 locks may be put to¬ 
gether by taking the pieces which come to 
hand.”* 

Whitney’s system did not achieve im¬ 
mediate acceptance. Captain Decius Wads¬ 
worth, later chief of the Ordnance Depart¬ 
ment, . . considered interchangeability 
pleasing to the imagination, rather than 
practically valuable.”** 

Nonetheless, by 1819, Major Dalliba, of 
the Springfield Armory, described hardened 
gages to pass on workpieces. In other words, 
acceptance of interchangeability had come 
quickly. In 1820, a government contract for 
20,000 pistols awarded to Simeon North 
specified that any part of any pistol fit any 
of the 20,000. 

It seems inconceivable that the system of 
interchangeability should have involved 
only Eli Whitney. More likely, as was the 
case with the development of machine tools 
in England, the seeds were there, ready to 
germinate. Although evidence is scant, it is 
believed that Simeon North was manufac¬ 
turing by the interchangeable method at the 
same time as Whitney, but at least by 1808. 
Roe suggests that this may have been 
so.*** Another source puts it more defi¬ 
nitely: “The Early Industrialist Simeon 
North, who was the first man to receive a 
contract for making government pistols 
(March 9, 1799), as well as the first to apply 
the principle of interchangeable parts in arms 
manufacture.”**** (Italics added.) This rev¬ 
elation proved interesting to the author 
since this same North family is part of his 
own lineage. 


*Guy Hubbard, “Development of Machine Tools in 
New England,” American Machinist. Series of 23 ar¬ 
ticles appearing from July 5, 1923 to September 18, 
1924. 

**Felieia Deyrup, Arms Makers of the Connecticut 
Valley, p. 87. 

***Joseph W. Roe, English and American Tool 
Builders, pp. 162-163. 

****Connecticut: A Guide to Its Roads, Lore and 
People, p. 112. 
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ARMS-MAKING, 
INTERCHANGEABILITY, 
MACHINE TOOLS 

England attempted to stifle the industrial 
life of the American colonies with its mer¬ 
cantilist policy. An example of this policy 
was the “Act of 1750”; its purpose was to 
maintain the colonies as suppliers of such 
raw materials as iron. Iron-working was 
even prohibited under penalty of a fine. 

Partly as a consequence of England’s re¬ 
strictions, machine tools were not even used 
before 1800 in the most important area for 
arms-making, the Connecticut Valley. By 
1830, in contrast with England, little head¬ 
way had been made with steam power. 

Whitney’s milling machine, Fig. 224, of 
about 1820—thefirstof its type—wascrude 
by English standards of the same date. 
Nevertheless, it was an important fore¬ 
runner in the rapid evolution of American 
machine tools. 

The circumstances that led to the adop¬ 
tion of interchangeable manufacturing and 
then to accurate machine tools are closely 
tied to arms-making. 

Frontier life stimulated Americans, al¬ 
ready demonstrating great independence of 
spirit, to achieve even greater self-suffi¬ 
ciency. New settlements were founded 
around dams used for water power. From 
this followed grist-mills and then saw-mills. 
Next came forging-shops and blacksmith 
shops. Indeed, early gunsmiths were closely 
allied with blacksmiths, so much so that 
many sons of blacksmiths turned to gun- 
smithing. 

Experienced gunsmiths accepted con¬ 
tracts and oversaw organized groups of 
workmen, each group making large lots of 
certain gun parts—a technique known as 
“division of labor.” Methods of making 
interchangeable parts were by pattern 
weapons (samples), swages and jigs, drill 
jigs and filing jigs. As Hubbard observes in 
“Development of Machine Tools in New 



FIG. 224 —Eli Whitney’s milling machine 
(about 1829 ) was crude by English 
standards of that date. It was, however, an 
important forerunner of machine-tools in 
America which immediately followed. 
Courtesy of New Haven Historical Society. 
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fig, 225 —The Colt Revolver, manufactured 
in Hartford, Connecticut. All parts were 
made interchangeable, enabling quantity 
production. This was known as the 
“American System." 

Courtesy of American Heritage, 
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England”: “Under this system it is evident 
that the quicker and more cheaply the con¬ 
tractor could do his work and still have it 
pass inspection, the more money he made 
and this was naturally a powerful incentive 
to the invention of improved tools, jigs, fix¬ 
tures and methods, as well as to the cultiva¬ 
tion of keen business and executive ability 
among the contractors.”* 

The next step in the movement toward 
interchangeable manufacturing was almost 
inevitable: the construction by armorers 
of special machines to manufacture parts to 
dimension. These machines were soon found 
adaptable to other processes, from which 
evolved the idea of basic machine tools. 

Spurred on by this new approach to man¬ 
ufacturing, schools of inventors, master me¬ 
chanics and master tool builders gradually 
emerged. A good example of how this trend 
took shape and mushroomed is seen in the 
Robbins & Lawrence Company, from which 
descended the Jones & Lamson Machinery 
Company. The firm traces to Windsor, a 
town which grew up around two dams in 
Vermont. It owes its origin to a gun made 
by Nicanor Kendall, the son of a black¬ 
smith. Kendall, Hubbard and Smith later 
began to manufacture rifles on an inter¬ 
changeable system, having barrels forged, 
bored and rifled by Eliphalet Remington, a 
skilled gunsmith in Ilion, New York. 

The system was so effective that they fin¬ 
ished one particular contract for rifles 18 
months ahead of schedule, an unprecedent¬ 
ed event for that time. This naturally led 
to other contracts; and by 1850, the firm, 
now known as the Robbins & Lawrence 
Company, was not only proficient in rifle¬ 
making, but had developed many advanced 
machine tool designs for their manufacture. 

Robbins & Lawrence rifles were shown 
in the Crystal Palace Exhibition in London 
in 1851. They could be disassembled and all 

*Guy Hubbard, “Development of Machine Tools in 
New England,” American Machinist. Series of 23 ar¬ 
ticles appearing from July 5, 1923 to September 18, 
1924. 
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parts interchanged. The official British re¬ 
port states: 

“Of these rifles, manufactured by Robbins 
& Lawrence, it is but just to say that they 
are among the best, if not the best, of any 
rifles manufactured in the world. . . 

These rifles drew the startled attention of 
such British engineering and military au¬ 
thorities as James Nasmyth, Joseph Whit¬ 
worth and the Duke of Wellington. At that 
time, all parts for British rifles were made 
by hand in different shops. They were then 
brought to the Tower of London, where 
each rifle was built up by laboriously filing, 
reaming and fitting individual parts. The 
result of this long and involved procedure 
were rifles that were not only crude, but 
non-uniform; no two were alike. 

Determined to look further into this new 
approach to manufacturing, an official Brit¬ 
ish commission was sent to America to study 
not only the plants of Robbins & Lawrence, 
but also those of Ames, Sharps and Colt, 
whose arms had also been displayed. The 
Colt company had been set up only three 
years earlier by Samuel Colt in Hartford, 
Connecticut, with the help of Root, a gifted 
mechanic. The company’s primary interest 
was that of capitalizing on the interchange¬ 
able manufacture of revolvers, and no ex¬ 
pense was spared in the facilities, Fig. 225. 

The most significant result of the Com¬ 
mission’s recommendations was a contract 
from the British government to the Robbins 
& Lawrence Company for virtually all the 
machine tools, 150 in number, as well as the 
tooling, to equip their new Enfield Armory. 

No two of the sample rifles supplied to 
Robbins & Lawrence from England were 
alike. Moreover, the gages were of wood, ac¬ 
curate to perhaps l/64th inch [0.4 mm]. 
The mechanics of the American firm made a 
master gun from these prototypes. Parts 
were kept uniform by accurate steel gages, 
hardened and carefully oilstoned to size. 

*Ibid„ p. 129. 


THE DEVELOPMENT OF 
STANDARD MACHINE TOOLS 

The machines built by Robbins & Lawrence 
for their own use form the basis of later 
standard machine tools, and their influence 
is clearly traced. One example is their plain 
milling machine, an improvement on Whit¬ 
ney’s miller. Closely copied by the George 
L. Lincoln Company, it became a standard 
in its line, and for many years thereafter 
was known as the “Lincoln Miller.” Pratt & 
Whitney was the next company to build the 
machine based on the Robbins & Lawrence 
prototype. 

Tracing the ancestry of the jig borer in 
his book Precision Hole Location,* Robert 
Moore begins with Wilkinson’s cannon- 

*J. Robert Moore, Precision Hole Location, pp. 
52-63. 


boring machine of 1776. He then goes on 
to mention a Universal Boring Mill built 
for Clarke in 1896, Fig. 226. As with the 
jig borer, this machine generated a hole 
around the machine spindle axis and lo¬ 
cated by coordinates. Moore comments: 

“The writer is not prepared to state that 
this is the first machine of this general 
design built; if not, it at any rate is un¬ 
doubtedly typical of its day. Its inspira¬ 
tion appears to have been derived from 
the Lincoln Miller.”* 

The interchangeable system spread to 
other manufacturing—such as that of clock 
making. In 1855, America was exporting 
400,000 clocks a year. As a result of this 
system, clocks were made so inexpensively 

*Ibid., p. 57. 



FIG. 226 —The Universal Boring Mill, first 
built around 1896, located by coordinates, 
and established the hole around the machine 
spindle axis as with a jig borer. However, 
it lacked the prerequisite accuracy to 
perform the functions of a jig borer. 
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fig. 227 — The Linear Dividing Machine, 
designed by Joseph Ii. Brown in 1850, 
graduated snales and steel rules for use by 
American industry. I is principle was kept 
secret until relatively recent times. 

Courtesy of Brown & Sharpe Manufacturing Co. 



that it was less costly to discard one than 
repair it. Many basic machine tools also 
evolved from the clock and watchmaking 
industry. 

Although by 1850 the United States was 
advanced in interchangeable manufacture, 
it was behind in certain other areas. Steel 
production, for example, was out of step 
with industrial progress. Initially, steel 
was derived from iron mined in Salisbury, 
Connecticut. However, demands could not 
be met and the quality failed to meet the 
user’s requirements. Consequently, by 
1855, the Springfield Armory was buying 
large quantities of steel from England and 
Scandinavia, to the consternation of Amer¬ 
ican iron-masters. The Colt company even 
preferred English steel, which was better 
refined and more homogeneous. 

DEVELOPMENT OF 
ACCURATE MEASURING 
INSTRUMENTS AND 
MACHINE TOOLS IN 
AMERICAN INDUSTRY 

Nowherewasindustrial backwardness more 
apparent in mid-nineteenth century Amer¬ 
ica than in the unavailability of adequate 
established standards. America, as observed 
in the “History of Official Length Stan¬ 
dards,” was of necessity closely tied to 
England in the matter of weights and 
measures because of political circumstance. 
There was also a lack of measuring tools 
and even simple steel scales. 

Charles Hildreth, superintendent of the 
Lowell Machine Shop, wrote: 

“The original standard unit of length 
adopted by the Lowell Machine Shop in 
1824 for use in the construction of ma¬ 
chinery is the English foot, according to 
a brass standard made by Cary of Lon¬ 
don, and preserved with care in the safe 
of the proprietors of the locks and canals 
of Merrimack River. This scale is un- 


FIG. 228— The Brown & Sharpe Universal 
Miller, a machine lo»l first produced in 
1860. 

Courtesy of Science Museum, London. 

British Crown Copyright. 
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fig. 22!) —L, Palmer’s Micrometer, French 
Patent No. 3762, 18U8. 

Courtesy of Science Museum, London. 

British Crown Copyright. 


dated, but has marked upon it, ‘Cary, 

London, Fahrenheit, 62 degrees’ 

[16.67° C].”* 

Two questions arise: 

1. Without “interchangeability,” how 
could the development of machine 
tools in England have been so rapid? 

2. Without recognized standards, how 
could interchangeability have reached 
such a high level in America? 

The answer to the first question is that in 
England parts were made not so much to 
dimensions as to fit. In reply to the second, 
there was strictly in-shop standardization 
in America. Initially, in neither England 
nor America was the need apparent for 
interchangeability of parts between one 
manufacturer and another. 

In England, the major proponent of stan¬ 
dardization of such elements as length stan¬ 
dards and screw-threads was Joseph Whit¬ 
worth. In the United States, two machine 
tool builders, Brown & Sharpe Manufac¬ 
turing Company and the Pratt & Whitney 
Company, were leaders in promoting and 
making available standards to industry. 

Brown & Sharpe 

Brown & Sharpe was founded in 1833 by 
Joseph R. Brown. By 1851, the company 
was manufacturing steel rules, scales and 
vernier calipers by using a linear dividing 
machine whose design features it long kept 
a secret, Fig. 227. It should be observed in 
passing that Brown’s competitor, a man 
named Darling from Maine, had also con¬ 
structed a dividing machine, and, like 
Brown, had his standards compared in the 
Coast Survey Office, Washington, D. C. Po¬ 
tential friction that might have developed 
was eliminated, however, when Darling 
joined forces with Brown & Sharpe. 

In 1851, only four vernier calipers had 
been sold. Although introduction of Brown 

*Ambrose Webster, “Early American Steel Rules,” 
American Machinist, April 12, 1894. 


& Sharpe measuring tools was slow, their 
impact was felt. At Colt, when the vernier 
calipers were first put into service, enough 
discrepancies showed up in their standards 
for Colt to temporarily favor a certain 
amount of “tailoring” in their fits. 

At the same time that Brown & Sharpe 
was making industrial measuring tools, it 
was also introducing important new ma¬ 
chine tool designs. Its Universal Miller of 
1860, Fig. 228, is considered a major accom¬ 
plishment. 

The “micrometer,” developed to settle a 
dispute on sheet metal gaging, was first 
presented to Brown & Sharpe in 1867 by 
Wilmot, of Bridgeport Brass. Lacking 
graduated markings, its potential was not 
then apparent. When Brown and Lucian 
Sharpe saw the “Systeme Palmer,” Fig. 
229, a true micrometer, at the Paris Expo¬ 
sition, they realized its commercial value. 
Shortly thereafter, the Brown & Sharpe 
company began to manufacture microm¬ 
eters in large quantities, Fig. 230. 

In the ensuing years, Brown & Sharpe 
built standards of even higher accuracy. In 




fig. 230 —First Brown & Sharpe one-inch 
Micrometer, 1877. Also shown are 
L. Palmer’s Micrometer and a modern 
Brown & Sharpe Micrometer. 

Courtesy of Science Museum, London, 

British Crown Copyright. 
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1878 they constructed a more accurate Mea¬ 
suring Machine, Fig. 231, largely the work 
of a Mr. Beale. Its basis was the “Standard 
Yard,” and had been compared with the 
standards of Washington. 

A letter from the Brown & Sharpe com¬ 
pany to J. E. Hilgard of the Coast Survey 
Office, Washington, regarding the metric 
standard in use by them at that time, 
reads: “Taking 39.370 as the standard, 
there is only 0.00023 inch [0.0058 mm] in 
the meter difference in our comparison, 
which perhaps is as close as may be ex¬ 
pected. We shall now consider your compari¬ 
son of our steel bar with the standard of 
Washington as correct, and in our compari¬ 
son with it we shall be able to detect errors 
as small as 0.000025 inch [0.0006 mm].”* 

Pratt & Whitney 

In AccuracAj for 70 years, published by the 
Pratt and Whitney Company, appears a 
statement that might at first seem pre¬ 
sumptuous, but understood in historical 
context, is not without justification. That 
statement is: "Pratt & Whitney established 
the inch.”** 

By way of background, Amos Whitney, 
a distant relative of Eli Whitney, and Fran¬ 
cis Pratt were first trained at Colt. They 
met at the Phoenix Iron Works, and while 
employed there set up a partnership to build 
machinery. At first they did so on a part- 
time basis. Later, when the partnership was 
operating successfully, they gave it full 
attention. From the start, they were com¬ 
mitted to accuracy, and were leaders in the 
establishment of standards. 

During the Civil War, Pratt & Whitney 
made rifles by the interchangeable method. 
The experience gained encouraged them to 
apply the concept to the manufacture of 
machine tools such as lathes, horizontal 
shapers, and the “Lincoln Miller.” 

*Luther D. Burlingame, “Pioneer Steps Toward 
the Attainment of Accuracy,” American Machinist, 
August 6, 1914. 

**Accuracy for 70 Years — 1860-1930, p. 35. 




Not long after 1870, Mr. Pratt secured a 
$350,000 order from the German govern¬ 
ment for machine tools. This was followed 
three years later with a $1,250,000 order 
for more. The German government later 
wrote: “The Pratt & Whitney Company 
has furnished the Royal Armories of Spon- 
dan, Erfurt and Donsitz with plants of ma¬ 
chinery which execute the work with such 
nicety and precision as to save one half the 
wages, and to render the government in no 
small degree independent of the power and 
skill of the workmen.”* 

In establishing its standards, Pratt & 
Whitney soon realized, however, that no 
*Ibid., p. 31. 


two standards in America were alike. Pro¬ 
fessor William A. Rogers, an astronomer at 
Harvard University, and Roger M. Bond of 
Pratt & Whitney set about establishing 
standards of their own in all areas of me¬ 
chanics. 

As J. Robert Moore observes: 

“The man of science turned his attention 
from the planets and the measurement of 
distances counted by millions of miles, to 
listen to the imprecation, perhaps, of the 
humble car repairer, lying on his back 
and swearing because a % inch nut—‘a 
leetle small’ —will not screw on a bolt— 
‘a trifle large.’ ”* 

*J. Robert Moore, Precision Hole Location, p. 62. 


Clarifications in the matter of length 
standards were made. For the “inch,” 
standard bars were compared to Bronze 
No. 11, thus indirectly related to the Im¬ 
perial Yard. For the “meter,” compari¬ 
sons were made to the working meter of 
the Conservatoire des Arts et Metiers, 
as well as to other bars which had been 
compared to the Metre des Archives. The 
result of three years of effort and thou¬ 
sands of dollars was the Rogers-Bond 
Comparator, Fig. 232. It was transported 
to the bureau in Washington to duplicate 
conditions of comparison; standard tem¬ 
perature was then 62°F. [16.67°C], after 
the English. 



fig. 232 --The Rogers-Bond Comparator 
was the result of three years’ icork by 
Professor William A. Rogers of Harvard 
University and Roger M. Bond of the Pratt 
& Whitney Company. 

Courtesy of Pratt & Whitney Co. 
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fig. 233 — (top) Swedish “Polhem Sticks,” 
preceding Johansson's invention, by at least 
150 years, enabled many different 
measurements to be made with one “gage.” 
Courtesy of Johansson Co. 



In a report of the American Society of 
Mechanical Engineers, Bond and Rogers 
were acknowledged for having “solved 
the problem of exact and uniform mea¬ 
surement.”* 

The Rogers-Bond Comparator be¬ 
came the basis of Pratt & Whitney Mea¬ 
suring Machines for years to follow. 






CARL EDVARD JOHANSSON 

Sweden can be credited not only with supply¬ 
ing some of the world’s finest quality steel, 
but also with giving it some of its greatest 
mechanics. One such mechanic contributed 
an innovation to measurement second only 
in significance to the concept of interchange- 
ability—namely the “gage block,” created 
in the 1890’s by Carl Edvard Johansson. 
Today, the term “gage block” has become 
almost synonymous with “measurement.” 

The importance of the gage block cannot 
be over-emphasized. On a practical level, it 
is one of a shop’s most useful length¬ 
measuring tools, and is more often than 
not its first direct representative of the 
official length standard. Where higher-level 
measuring accuracy is concerned, as is the 
case with the national bureaus, it is usual¬ 
ly somehow related to the gage block. 

Much has been written about the gage 
block, but no greater insight can be gained 
into its significance than by following 
Johansson’s footsteps as he developed it. 

Eskilstuna Rifle Factory- 

Arms manufacturing in Sweden, as was the 
case in other countries, stimulated interest 
in greater precision, which in turn led to im¬ 
portant mechanical development. In Swe¬ 
den, as well as the rest of the world during 
the 1880’s, the single most important item 
produced in quantity— even surpassing the 
clock and the sewing machine— was the rifle. 

The principle of interchangeable manu¬ 
facture was known in Sweden for some time, 


*Joseph W. Roe, English and American Tool 
Builders, p. 182. 


fig. 23 ! —■ (middle) Johansson’s Gage with 
measurement stud held in hand, illustrating 
size and use. 

Courtesy of Johansson Co. 


fig. 235 -ffibollom) A small gage of steel 
with measurement studs, allowed several 
sizes in one bar. Made by Johansson, it 
was one step toward his concept of the 
gage block set. 

Courtesy of Johansson Co. 
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fig. 230 —Using a special cannier weighted 
micrometer, Johansson was able to measure 
his gage blocks with a constant force. 
Courtesy of Johansson Co. 


having been introduced by Hilge Palm- 
crantz. Thus in the Eskilstuna rifle factory, 
where the Remington rifle was then manu¬ 
factured, “fixed gages” were used. In fact, 
there was only one Brown & Sharpe mi¬ 
crometer in the whole Eskilstuna plant. 
The rest of Sweden and other countries 
still relied upon in-shop standardization. 

To understand the emergence of the gage 
block requires knowledge of its predeces¬ 
sors, beginning with the “Polhem Stick.” 
“Polhem Sticks,” Fig. 233, first made of 
wood and later of iron, were an old princi¬ 
ple; they enabled many separate measure¬ 
ments with one “gage.” 

Johansson, an apprentice at Eskilstuna 
who quickly became its chief armorer, knew 
the “Polhem” method of measurement, Fig. 
234. It is known, furthermore, that he fash¬ 
ioned a small gage of steel with measure¬ 
ment studs, Fig. 235, which allowed for sev¬ 
eral sizes in one bar. This gage stands as the 
first step toward his significant invention. 

The Eskilstuna plant was then planning a 
change from the Remington rifle to the 
German Mauser. The prototype of this 
model was made to Mauser’s specifications 
at Pratt & Whitney. The Mauser was a 
more complex gun, requiring higher accura¬ 
cies. J ohansson was appointed to the com¬ 
mission which went to the Mauser plant in 
Germany to study their methods. On his re¬ 
turn, Johansson pondered the overwhelm¬ 
ing task of duplicating for the Eskilstuna 
factory the thousands of gages he had seen 
at the Mauser works at Oberndorf. 

Before his return, he conceived of the 
idea of a set of gage blocks, 102 in num¬ 
ber, that would enable him to make 20,000 
different combinations of measurement. 

In 1875, Professor Tyndall demonstrated 
to the Royal Institution in England that 
Whitworth’s polished plates would adhere, 
and that the force of adherence was greater 
than atmospheric pressure. Whether Jo¬ 
hansson was at first aware of this phenome¬ 
non is not certain. In any event, the fact 


that finely lapped gage blocks adhere, or 
“wring,” madehis system both possible and 
practical. 

The preliminary work for the first set of 
blocks was done in the rifle factory. For the 
final finishing, however, Johansson worked 
nights at home, often aided by his wife. 
Lapping was accomplished with a modified 
sewing machine, and measuring with the aid 
of a special counter-weighted micrometer 
(for consistent gaging force), Fig. 236. 
Blocks were stress-relieved for stability, 
probably by exposing them to alternate 
heat and cold. The entire operation was 
kept secret, and all tools were put carefully 
away and left behind locked doors. 

The Sandvik Steel Company of Sweden 
furnished the steel (chrome-nickel bar steel) 
used for Johansson’s blocks, and all heat 
treatment was overseen by Peterson, Jo¬ 
hansson’s chief metallurgist. 

To achieve stability in the gage blocks, 
they were heated, then allowed to return to 
room temperature. Next, they were deep- 
frozen and again allowed to return to room 
temperature. This cycle was repeated nine 
times. Johansson recounted his theory of 
stress-relieve in a metaphor that is uncan¬ 
nily close to modern concepts: 

“The molecules in the steel are like little 
children. When they get warm, they dance 
and jump around. When they get cold, 
they quiet down and go to sleep. After 
you wake them up and put them to sleep 
enough times they get tired, and the last 
time they stay asleep. In this manner the 
gage blocks are made stable and “stay.”* 

Commenting on Johansson’s shop tech¬ 
nique, particularly his sense of the practical, 
his biographer, Althin, observes: “. . . in all 
his designs he avoided working with gauges 
that had to be adjusted according to a grad¬ 
uated scale.”** 

‘Author’s conversation with Gilbert L. Dan- 
nehower circa May 1, 1969. 

“Torsten K. W. Althin, C. E. Johansson 186A- 
193A: The Master of Measurement, p. 105. 




FIG. 237 — Johansson’s first set of gage- 
blocks to be sold. His principle ■was to 
revolutionize the concept, of length 
measurement. 

Courtesy of Johansson Co. 
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For the first gage set, only those stan¬ 
dards readily available were used for cali¬ 
bration of his blocks. This set alone was 
the basis for all measurements of the Mauser 
6.5 mm rifle. 

As he gained confidence in his system, he 
gathered more assistants for the manufac¬ 
ture of what had now become a product, 
Fig. 237, finally resigning his position to 
devote full time to the new concept. 

The slow acceptance of his idea and his 
dogged determination find expression in a 
letter Johansson sent to his father in 1899: 
“I do not think they understand the benefit 
of my measurement system. All the same, it 
is a thing of the future, and I will have it 
recognized.”* 

The next step Johansson took was more 
crucial. The motivation for this step most 
likely relates to his inability to secure a pat¬ 
ent protecting a system which could be eas¬ 
ily copied. As a result, its uniqueness could 
only be established by what would today be 
called “traceability.” 

Johansson constructed a 100 mm gage 
[3.937 inch]. Its length was established by 
comparison to several standards, including 
one by Brown & Sharpe. The gage was 
brought to the Swedish Bureau, where the 
Chief of Office compared it to their standard 
at 20°C. [68°F.], a temperature considered 
a mean. The reaction of the Chief is an 
amusing commentary on those “warm¬ 
blooded” individuals who want a new tem¬ 
perature-standard above this chilly figure: 
“20°C.? We must fire up to get that tem¬ 
perature.”** 

When Johansson calibrated his gage at 
the International Bureau at Sevres, France, 
he had almost reached his goal of traceabil¬ 
ity. A few discrepancies arose, however, be¬ 
cause his blocks were lapped at 20° C. 
[68° F.], while the BIPM was then using 
0°C. [32° F.] as a standard. A new rod was 

*Torsten K. W. Althin, C. E. Johansson 186]>- 
1931+: The Master of Measurement, p. 70. 

**Ibid., p. 74. 


made, set at 0°C. [32°F.], and in April, 
1903, it was sent to the BIPM. In Johans¬ 
son’s letter to a friend, Ternstrom, who ar¬ 
ranged for the testing, he said: 

“If the people of the Bureau measure cor¬ 
rectly, they shall find that each block lies 
within 0.001mm [40 millionths of an inch] 
limit of accuracy in relation to the 100 
mm [3.937 inches] measure.” 

He further added: 

“If they arrive at any other result, then 
they had better adjust their measuring 
appliances.”* 

Johansson’s gage was successfully cali¬ 
brated by the BIPM, establishing “trace- 
ability.” His long-sought goal was finally 
attained. 

Johansson then proceeded to promote his 
gages throughout the world, Fig. 238, in¬ 
cluding the U.S.A., where he was aided by 
his brother, Arvid. He also went to the Na¬ 
tional Physical Laboratory in Teddington, 
England, where Dr. Glazebrook, its head, 
enthusiastically accepted the “gauge block 
set.” 

The BIPM measured Johansson’s blocks 
over a period of several years by many dif¬ 
ferent techniques, including interferometry. 
At first, they publicly stated that the ac¬ 
curacies quoted by Johansson were impos¬ 
sible. Benoit of the BIPM, said to Spang- 
berg, who promoted Johansson’s system: 

“Young man, if you had any idea of the 
enormous difficulties we are confronted 
with in our laboratory in reading with 
certainty a micron [40 millionths inch], 
you might perhaps speak a little less 
lightly of this high grade of precision.”** 

Benoit later became a foremost advocate 
of Johansson’s system, and in a summary of 
the BIPM’s findings reported: 

“One cannot get away from the finding 
that the accuracy of these gages is re- 

*Ibid., p. 81. 

**Ibid., p. 126. 


markable, so for one and the same magni¬ 
tude it amounts to the degree of accuracy 
of the interference measurements, and 
adjustment only displays errors less than 
one-tenth of a micron [0.000004 mil¬ 
lionths of an inch].”* 

Commenting on the success Johansson 
enjoyed in France, Althin notes: 

“It was natural for the French character 
to understand the clear logic of the Jo¬ 
hansson system, and French scientists 
were the first to recognise the wide im¬ 
portance of it.”** 

By 1907, improvements in finishing the 
blocks enabled them to be wrung together 
with a force of up to seventeen atmospheres; 
and by 1916 with up to thirty-three atmo¬ 
spheres. 

In 1914, an American, Gilbert L. Danne- 
hower, became fascinated by the Johansson 
system while still in technical college. After 
graduation, he worked his way to the posi¬ 
tion of being Johansson’s Sales Manager in 
the U.S. At first, Dannehower encountered 
difficulty explaining the use of the blocks, 
so he organized a sales force consisting of 
twelve former inspectors and toolroom fore¬ 
men. Nevertheless, a common reaction of 
prospective customers on being informed of 
a $900 price tag for the set was: “What are 
they made of, gold?” This kind of reaction 
was not that of men of vision. Commented 
Charles Kettering when the system was pre¬ 
sented to him at the Delco plant in Dayton, 
Ohio: 

“Gentlemen, this is the finest measuring 
tool in the whole world. The sooner we 
learn to use it, so much the sooner we can 
stop ‘fitting’ parts together and simply 
‘put’ them together. The day is now not 
far off when every toolmaker will have to 
have a set of these blocks at his elbow, or 
he just won’t know what the hell he is 
doing. Let’s buy ten sets and get them 

*Ibid., p. 86. 

**Ibid„ p. 89. 
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fig. 23S — The gage block concept was not 
accepted without salesmanship. Johansson 
lectured throughout the world to promote the 
idea of h is gage blocks. 

Courtesy of Johansson Co, 


fk;. 239 —Facilities and personnel of the 
NBS were devoted toward aiding industry 
during World War I. 

Courtesy of the NBS. 
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fig. 24.0 — G. Dannehower stayed at the 
Johansson home in Bob:iken, Sweden, 
several tunes during World War l while 
securing gage blocks for the United States. 
Courtesy of (3. Dannehower. 


FIG. 241 — G. Dannehower recalls with great 
affection the memory of his friend the 
“Eskilstuna Blacksmith ,” as Johansson 
modestly referred to himself. 

Courtesy of G. Dannehower. 



out in the toolroom and inspection de¬ 
partments!”* 


During the First World War, the unavail¬ 
ability of “Jo Blocks” from Sweden, be¬ 
cause of German submarine blockades, 
caused a near-crisis in the United States, 
then girding industry for war-time produc¬ 
tion, Fig. 239. Dr. Stratton, Chief of the 
National Bureau of Standards, realizing the 
critical importance of gage blocks, got 
Dannehower released from army duty, 
whereupon he was immediately sent to 
Sweden. He was instructed to “stay there 
and get blocks made in the English system 
and bring them home.”** 


*Unpublished letter from Gilbert L. Dannehower, 
April 15, 1969. 

**Idem. 


Dannehower went to live with the Jo¬ 
hansson family in Eskilstuna (Figs. 240 
and 241) where work quickly proceeded to 
produce inch gage blocks. A total of 128 
sets were made after four weeks. Danne¬ 
hower managed to smuggle the blocks 
aboard ship on his return to the U.S., despite 
the presence of German inspectors. He was 
apprehended, however, by Canadian offi¬ 
cials and upbraided by Jacobsen, the presi¬ 
dent of the Norwegian-American Line, for 
risking ship and passengers with contraband. 
The matter was soon cleared up when 
Dr. Stratton was contacted by the officials. 

Dannehower made two additional trips 
“. . . to keep supplies coming . . . and mass- 
production and interchangeability was pos¬ 
sible, because we had our supply of blocks,” 
as he describes it. When the War was over, 
Henry Ford observed: 

“This must not happen again; we must 
have the secret methods within our coun¬ 
try’s boundaries. I will buy the manufac¬ 
turing rights.”* 

In 1923, Johansson became associated 
with Henry Ford, applying his skill and 
energy to the greatest single product of 
mass-production through interchangeabil¬ 
ity—the automobile. 

During the war years, all-out efforts were 
made to produce gage blocks in the U.S. At 
this time, Major Hoke, an inventor, came 
to the National Bureau of Standards with 
an idea of making gage blocks on a large 
scale. The project was somewhat success¬ 
fully pursued over several years as an impor¬ 
tant part of the war effort, Fig. 242. Lat¬ 
er, the “Hoke” blocks were taken over on 
a mass-production basis by the Pratt & 
Whitney Company. 

Since that time, there have been count¬ 
less worldwide entrants into the field of 
gage-block manufacture, attesting to their 
widespread, indispensable use. 


*Idem. 
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FIG. 242 — “Hoke" blocks were manufactured 
at the NBS during World War I to help 
supply U.S. industry’s urgent need for 
length-measuring standards. 

Courtesy of the NBS. 


EVOLUTION OF COORDINATE- 
LOCATING MACHINE TOOLS- 
JIG BORER 

The jig borer was not the earliest, nor is it 
today the only machine tool to position by 
means of coordinates. Nonetheless, it is 
recognized as the most accurate of hole- 
locating machine tools. Certainly its in¬ 
troduction was a major step forward in 
the hole-location process, itself a key to in¬ 
terchangeability. 

Every standard machine tool underwent 
a long evolution before reaching its most 
current design. Some of the ancestors of the 
jig borer have already been mentioned. 

The jig borer combines two essential 
functions: 

1. Single-point boring, meaning that the 
tool generates a hole as it is rotated 
about its own axis. Wilkinson’s bor¬ 
ing mill is a good example, Fig. 217. 

2. It established location by means of co¬ 
ordinates, as in Clarke’s horizontal 
boring mill, Fig. 226. 

Yet a machine tool that would accurately 
locate and bore, the two requisites of a jig 
borer, were very late in coming. In Precision 
Hole Location, J. Robert Moore details the 
problems inherent in hole location in a chap¬ 
ter entitled “They Got Holes In Somehow.” 
He furthermore examines why hole loca¬ 
tion equipment was so late in development. 
The chapter’s title suggests that the task 
was still a craft and not an engineered 
solution.* 

In the clock industry, for example, meth¬ 
ods of establishing holes correctly spaced re¬ 
quired laborious processes, such as laying 
out with a scriber, prick-punching, drilling, 
reaming, wiggling, buttoning, etc., Fig. 
243. As evidence that these methods could 
position at least more accurately than exist¬ 
ing machine tools, Joseph Woodworth, au- 


*J. Robert Moore, Precision Hole Location, pp. 
15-51. 




FIG. 243 —Prior to the jig borer, accurate 
hole location such as required in the clock 
industry was established by transferal 
methods. Depicted is a hole being bored in 
the master plate after the model Itas been 


“wiggled up” and removed. When the 
master plate is finished, it is used as a 
reference for positions of all holes in tools 
used to make finished parts for clocks. 
Precision Hole Location. 


159 













fig. 244 — Early “Machine a Pointer" (jig 
borer), the Dixi company, date given as 
1905. in initial design, holes were first 
“pointed" or punched, then removed from 
the machine for drilling. 

Courtesy of Dixi. 


FIG. 245 —Jig borer, the SIP 

(Socieie Genevoise d’Instruments Physique) 

company, produced about 1912. 

The similarity between the SIP and Dixi 
designs, apparently produced independently, 


is striking. Later, the “pointing machine” 
was fitted with a spindle to enable 
machining holes in place. 

Courtesy of SiP. 



thor of American Toolmaking and Inter¬ 
changeable Manufacture, stated in 1911: 

“There is only one method for locating 
bushing holes in small and medium-sized 
jigs accurately and expeditiously . . . and 
this is known as the button method.”* 

Switzerland—Machines a Pointer, 

SIP company and Dixi company 
Shortly before World War I, several inde¬ 
pendent efforts were leading toward the 
creation of today’s jig borer. The Swiss, 
long famous for clock and watchmaking, 
appreciated precision, fine mechanisms and 
instruments. As in the United States, the 
“master plate” and other transfer methods 
were accepted as the only way of establish¬ 
ing interchangeability, such methods dating 
back over 100 years. Parts were thus made 
alike, but not to dimensions. 

The Societe Genevoise d’ Instruments 
Physique (SIP) was founded in 1860, in 
Geneva, Switzerland. Geneva was already a 
center of scientific, scholarly research. Com¬ 
posed primarily of instrument-makers, SIP 
introduced a small instrument in 1912 “de¬ 
signed to accurately measure distances and 
angles between a fixed point and other 
points located anywhere around it.** This 
instrument was created at the request of 
watchmakers, who needed a device for plac¬ 
ing holes in exact position. 

It might be mentioned that Dixi, S.A. of 
Le Locle, Switzerland, had earlier (about 
1905) designed an instrument for a purpose 
similar to that of SIP. The similarity be¬ 
tween the designs is striking, Figs. 244, 
245. Location was established by coordi¬ 
nate movement, but in true watchmaking 
fashion the holes were first center-punched, 
then removed for drilling. Later, this 
“pointing machine” (the French term for 
jig borer is still Machine k Pointer) was 

*Joseph V. Woodworth, American Toolmaking 
and Interchangeable Manufacture, p. 45. 

**Burnham Finney, “Coming: The Decade of 
Metrology,” American Machinist, July 9, 1962. 


fig. 24(1 — Modern SIP Jig Borer 
( planer-type) adapted to numerical control. 
Courtesy of SIP. 
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kg, 247 —Modem Dixi Jig Borer 
(h orizontal spindle-type ). 

Courtesy of Dixi, 


fitted with a drilling spindle, and holes were 
cut in place, saving time of removal from 
the machine and enabling greater accuracy. 

At first, lead screws with compensators 
for lead error were used. The principle of 
the scale and microscope was later adopted 
by both firms. More modern jig borers of 
SIP (planer-type) and Dixi (horizontal 
spindle) are shown in Figs. 246, 247. 

Pratt & Whitney Jig Borer, 1917 

The Pratt & Whitney Company, mean¬ 
while, was in the process of equipping vari¬ 
ous foreign arsenals for the manufacture of 
arms. Required were large quantities of jigs, 
fixtures and gages where holes had to be 
drilled, reamed, or bored to pre-determined 
location. 

Since existing methods, such as “button¬ 
ing” of jigs, were too slow, Pratt & Whitney 
constructed a machine with a vertical spin¬ 
dle which would bore to location. The result 
reflected the Company’s machine tool tra¬ 
dition. Conceived by a Swede, B.M.W. 
Hanson, the Pratt & Whitney “YigBorer,” 
as he referred to it, was, compared with the 
original Swiss versions, more a machine tool 
than an instrument, Fig. 248. 

Drawing upon its 30 years’ experience 
with measuring machines using both end 
standards and line standards, Pratt & Whit¬ 
ney decided on end measures for even spac- 
ings and a micrometer and dial for subdivi¬ 
sions. Because of its dual origin, it can be 
understood why the jig borer still retains 
the characteristics of both machine tool 
and instrument. 

From the 1930’s on, the jig borer re¬ 
ceived rapid acceptance. Since it was a pre¬ 
cise and expensive machine, many felt at 
first that the jig borer should be made mas¬ 
sive enough to handle from the smallest to 
the largest size workpieces. This concept 
naturally increased the cost of using a jig 
borer. As a result, it was not unusual to 
have only one jig borer servicing an entire 
geographic area, with attendant prestige to 




PIG. 2-18- -The jig borer, produced by 
Pratt & Whitney, 1917, was designed from 
the beginning to hare a boring spindle. 
Courtesy o f Pratt & Whitney Co. 
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fig. 249 —The Moore Special Tool 
Company produced it s first, jig borer 
(Model No. 1 ) in 1932. The machine was 
small, fast and employed hardened, ground 
and lapped lead screws. Richard F. Moore, 
founder and president with the first jig 
borer. 



its owner. All precision hole location work 
had to await the time when it could be put 
through his jig borer. 

Naugatuck Valley 

It has been previously mentioned (pages 
149-150) that America very early adopted 
interchangeability in the manufacture of 
clocks, and shipped large quantities to Eu¬ 
rope. Roe traces the development from 
buttons to brass, to clocks and watches, con¬ 
centrating on the important clock-making 
area of Naugatuck Valley.* 

Moore Jig Borer 1932 

A small firm, the Moore Special Tool Com¬ 
pany, was founded in Bridgeport, Connecti¬ 
cut, in 1924, specializing in highly accurate 
gages and dies. Much of its early work was 

*Joseph W. Roe, English and American Tool 
Builders , pp. 231-238. 


for the clock and brass shops of the Nauga¬ 
tuck Valley. By 1932, Richard F. Moore 
had designed and built a jig borer for his 
firm’s own use, and then for sale to “adapt 
to toolrooms and jobbing shops making 
tools and dies for clocks, locks, small elec¬ 
tric devices, typewriters, adding machines, 
toys, instruments and many other small 
and moderate-sized parts.” He had set for 
himself the requirement that “the price 
should be low, so that the machine will pay 
for itself in a short time.”* 

The No. 1 Moore jig borer was small, ac¬ 
curate and fast. Its great innovation was the 
use of hardened, thread-ground and lapped 
lead screws, which gave greater security to 
measurement than the soft, compensated 
ones. In a small machine, the lead screw 
provided many advantages over either a 
scale or end measure, most notably in faster 
setting. The simplicity of design enabled the 
machine to be offered at a reasonable price, 
and it soon achieved acceptance. This con¬ 
tributed to the jig borer becoming a basic 
machine tool. 

Moore Jig Grinder, 1940 

In 1940, Moore invented the jig grinder, 
Fig. 250. This machine tool overcame the 
problem of having to jig bore parts in a soft 
state and somehow trying to compensate for 
the resultant hardening distortion. Now with 
the jig grinder, holes could be ground to loca¬ 
tion after hardening, a capability especially 
needed for such accurate work as progres¬ 
sive motor-lamination dies, watch and clock 
dies, and hardened jigs, fixtures and gages. 

By 1946, the National Cash Register 
Company had specified that all dies, where 
accurate hole-location was involved, had to 
be “jig-ground.” From today’s perspective, 
this would seem a logical requirement, but 
considering the then recent introduction of 
the jig grinder, such a specification required 
great perception by management. 

*Private conversation with Richard F. Moore, 
founder and president, Moore Special Tool Co. 


By 1950, Moore had introduced “con¬ 
touring” features to the jig grinder in a 
patented design, Fig. 251. So revolutionary 
was the contouring feature as applied to die¬ 
making that Frederick C. Victory and Rich¬ 
ard F. Moore felt it necessary to devote a 
part of their book, Holes, Contours, and Sur¬ 
faces, to an explanation of the process.* 

At this time, the jig borer was also en¬ 
larged and improved, Fig. 252. 

Without doubt, the basic locating fea¬ 
tures of the jig grinder evolved from, and 
are identical with the jig borer. Yet the 
jig grinder is considered by some to be 
more accurate or more sophisticated than 
the jig borer. This may possibly be attrib¬ 
uted to the higher finish achieved with 
ground surfaces. 

Introduction of the Universal 
Measuring Machine 
The writer is often asked, particularly by 
jig borer owners: “Why is the Universal 
Measuring Machine necessary at all, given 
the accuracy of today’s jig borers? Isn’t my 
jig borer sufficient for any measuring 
task?” The answer may be found in the na¬ 
ture of the jig borer itself, since it evolved 
toward a design which seeks the ultimate in 
positioning accuracy. Certainly, one can 
measure with the jig borer. Indeed, the 
very function of boring to accurate location 
involves the measuring process. The most 
accurate inspection which can be made of a 
jig-bored part, outside of a Universal Mea¬ 
suring Machine, is right on the jig borer 
after machining. 

The jig borer does have limitations, how¬ 
ever, notably in its column, housing and 
spindle design. Although the spindle must 
be accurate for measuring purposes, it must 
also be capable of taking a substantial ma¬ 
chining cut. Therefore, weighed in with the 
requirement for accuracy are the addition¬ 
al requirements of rigidity, reliability and 

*Frederick C. Victory and Richard F. Moore, 
Holes, Contours and Surfaces, pp. 139-159. 
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PIG. ?.50—The jig grinder (Model No. 1) 
invented by Moore in 191,0. The term “jig 
grinder" was selected to imply the machine’s 
jig borer ancestry. It enabled holes to be 
ground to location, including taper, 
after the piece was hardened. 


FIG. 251— The No. 2 Jig Grinder- 
revolutionized die-making by enabling 
contours (portions of radii , both male and 
female) to be ground to location. 


pig. 252 —The No. 2 Jig Borer utilized the 
same principles as the Model No. I Jig 
Borer, but was larger overall , with an 
infinitely variable spindle speed; moreover 
it introduced the use of Nitralloy lead screwi 



No. 3 Measuring Machine 


No. 3 Jig Borer 


No. 3 Jig Grinder 


fig. 253 — The No. 3 Universal Measuring the Model No. 3 Jig Borer, fig. 254, and Jig Grinder , fig. 255. 

Machine successfully tested the principle 

of hardened, ground and lapped double 

V-'Ways, as well as oilier advanced features 

of the Model No. 3 design. Once proven, 

these new features were incorporated into 
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fig. 250 —Coordinate Measuring Machine 
(CMM’s) measure hole locution many 
times faster than a Universal Measuring 
Machine. However, they are less accurate 
and of more limited application. 



COORDINATE MEASURING MACHINE (CMM) 



serviceability. Another unavoidable factor 
to be taken into account in using a jig 
borer is the presence of heat-producing 
elements. Because of the high RPM’s in¬ 
volved, the spindle bearings have a tend¬ 
ency to warm up. Moreover, heat from 
motors, gears, belts and pulleys impose cer¬ 
tain modifications in design. 

One distinguishing modification to be 
noted is the use of a cast Invar spindle hous¬ 
ing (a costly 36% nickel alloy) with No. 3 
Jig Borers and Jig Grinders. It is used not 
so much to prevent heat, but to minimize its 
effect on accuracy. A measuring machine 
spindle housing of cast Invar, on the other 
hand, would be an improper and un¬ 
necessary use of this material. 

Since errors are introduced during the 
machining operation, holes cannot be put in 
as accurately as their location can be mea¬ 
sured. 

In contrast, the design of the measuring 
machine is concentrated around the singu¬ 
lar function of measuring. As a result, it is 
more accurate and more efficient. It is natu¬ 
ral that the measuring machine should 
“borrow” those design principles that con¬ 
tribute to accuracy in a jig borer. However, 
just as the jig grinder is a distinct type of 
machine, though it “borrows” from the jig 
borer, so too is the Universal Measuring 
Machine. It is quite understandable, thus, 
that Dr. Farago would describe this ma¬ 
chine as a “Jig Borer-type Measuring Ma¬ 
chine.”* 

Figs. 253, 254 and 255 show the No. 3 
Measuring Machine, No. 3 Jig Borer and 
No. 3 Jig Grinder respectively. Base con¬ 
struction is virtually the same; differences 
are principally in the nature of column and 
spindle construction plus the three-times 
greater accuracy of the Measuring Machine. 

Another reason for the emergence of the 
Universal Measuring Machine is the fact 
that the jig borer came to be increasingly 

*Francis T. Farago, Handbook of Dimensional 
Measurement, pp. 284, 298-306. 


used in inspection processes. Some were 
used totally for that purpose. It was inevit¬ 
able, therefore, that demand would arise 
for a machine which would just measure and, 
with all the controls and capabilities built 
in, do the measuring j ob more efficiently. 

A further reason for increased use of the 
Measuring Machine relates to the need to 
remove special, ultra-accurate parts from a 
machining environment into an environ¬ 
ment devoted exclusively to inspection. 
Here a “fresh look” can be given the part in 
a clean, temperature-controlled environ¬ 
ment by a machine designed strictly for 
measuring and used by personnel with a 
strong bent for inspection. 

Taper-Probe Measuring Machine 
(CMM) 

The so-called “Coordinate Measuring Ma¬ 
chines,” Fig. 256 are of a different evolu¬ 
tion. They seem to have emerged out of the 
need for checking parts made in production 
to thousandths of an inch tolerance [several 
hundredths of a mm]. Prior to their intro¬ 
duction, it was often the case that it took 
longer to measure a part than to produce it. 
This was an intolerable condition with pro¬ 
duction parts. 

Coordinate Measuring Machines are use¬ 
ful for measuring parts with great speed. 
They can measure many times faster than 
would be possible with a Universal Mea¬ 
suring Machine. Their stated accuracies 
are to several ten-thousandths of an inch 
[a hundredth of a mm]. Dr. Farago lists 
its discrimination at 100 micro inches [2.5 
Aim].* 

These machines must make certain com¬ 
promises to attain speed in picking up a 
hole. No rotatable spindle, as such, is used, 
but a vertical slide carries a taper-probe 
which seats in the hole. Since only the top 
rim is picked up, which measurement might 
include a burr or a chamfer, they do not 

*Ibid., p. 284. 


fig. 257— The taper-probe of a CMM 
records hole location quickly but does not 
truly establish the axis of the hole, nor 
analyze hole “geometry.” 
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fig. 258 —The precision scale is an 
important and convenient form of length 
standard. Its accuracy-potential does not 
equal that which can be achieved with an 
end standard, now that the ultimate 
standard is the wave length of krypton 86. 



truly measure the axis of the hole, Fig. 257. 
Errors in hole “geometry” can therefore go 
undiscovered. Reliability of measurement is 
also dependent on the accuracy and con¬ 
centricities of the various style probes 
used. A study of their construction reveals 
that no real geometric accuracies can be at¬ 
tained, and none are usually given.* 

ESTABLISHMENT OF MOORE 
LINEAR STANDARDS 

The level of tolerance in the early days of 
the jig borer was to about ± .0002 inches 
[ ±0.005 mm). Such a tolerance allowed suf¬ 
ficient margin to use the best commercially 
available gage blocks and end standards as 
final length masters. A Universal Measuring 
Machine, as conceived, made such length 
standards no longer adequate. 

In 1950, Moore started to upgrade its 
linear standards in preparation for coordi¬ 
nate-locating machines to work to mil¬ 
lionths of an inch [hundredths of a /im]. 
There was a vast difference between the 
requirements for “tenths” [several /xm] 
and that of “millionths” [hundredths of a 
Atm]. 

Hand-lapped Step Gages of 1 inch incre¬ 
ments [30 mm steps in metric Step Gages] 
were in use, but they were not accurate 
enough. It was at once apparent that the 
first requirement would be to secure stan¬ 
dards the lengths of which were known to a 
few millionths of an inch [a few hundredths 
of a Atm]. 

The first gages obtained were a sixteen- 
inch gage block and 16 separate one-inch 
blocks. It was intended that the ends of the 
inch-blocks would be tied together, but it 
was quickly learned that gage blocks were 
made to be used vertically and not to be 
fastened together for horizontal use. The 
following limitations of these gages became 
apparent from this initial effort: 

’“Measurement: quick vs. close,” American Ma¬ 
chinist, July 3, 1967. 


1. The support surfaces were not flat, 
and the gages would bend. 

2. The ends were not square to the sup¬ 
port surfaces. 

3. If screwed together, the gages were 
compressed. 

4. If the gages are wrung together, there 
is no true support; not to neglect the 
wringing film of l /i millionth of an 
inch [0.000006 mm] per wring and the 
added uncertainty when that many are 
wrung together. 

5. The 16-inch [metric end standard is 
480 mm] gage block accuracy was sim¬ 
ply not there. 

6. The gages could not be compared 
adequately. 


Moore next secured gages with squared 
ends, so that they could be laid down. The 
micro-finish of the gages was high, but the 
accuracy was only .0001 inch [0.0025 mm]. 
Moreover, the square ends were not parallel 
and flat. Efforts to rework the gages were 
not successful, since the unsymmetrical 
ends could not be lapped accurately. 

In 1955, a precision scale of “H” section, 
exceptionally high in quality and workman¬ 
ship, Fig. 258, was purchased. Serious 
thought was given to making this the final 
reference standard, especially in view of the 
dissatisfaction with previous standards se¬ 
cured. Reading repeatability of the scale, 
however, was only about ± 20 millionths of 
an inch [0.0005 mm]. Even this dropped off 
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fig. 259 -Chart records the calibrations by 
identical means of two Step Gages hi the 
laboratories of Moore, the NBS and NFL. 
Through use of a No. 3 Universal 
Measuring Machine, overall length of a 
known 18-inch standard [the master metric 


end standard is l t 80 trim] was transferred 
to the 18-inch [ f80 mm with metric] 
increment of the Step Gage. The remaining 
steps were secured by a process of sub¬ 
division. 


CALIBRATION OF MOORE 18" STEP GAGE BY MOORE, 
Unit: 0.000001 (1 millionth) inch 

, NPL and NBS 

Inches, zero to 

Gage No. 54 

Gage No. 54 

Gage No. 55 

Moore 

NPL 

NBS 

1 

0 

0 

0 

2 

0 

+ 1 

+ 1 

3 

0 

0 

-1 

4 

0 

-1 

4-3 

5 

0 

+ 1 

+ 5 

6 

0 

0 

+ 5 

7 

0 

0 

+ 7 

8 

+ 1 

-1 

4-3 

9 

0 

0 

+ 3 

10 

0 

0 

4-4 

11 

-1 

-1 

+ 6 

12 

-1 

+ 1 

4-7 

13 

0 

-1 

+ 7 

14 

+ 1 

0 

+6 

15 

0 

-1 

4-2 

16 

-1 

+ 1 

+ 2 

17 

0 

+ 4 

+ 1 

18 

0 

+2 

-3 



due to eye fatigue, especially during pro¬ 
longed microscope use. Two other factors 
were disturbing: 

1. Working to the calibration chart was 
not only inconvenient, but risky. 
There was a danger that the correc¬ 
tion factor be used with the wrong 
sign. This was not felt to be a desira¬ 
ble system as part of a production op¬ 
eration. 

2. “Accuracy of determination” of a 
scale by the national bureaus was low. 

About this time, Frederick C. Victory, 
then Chief Engineer, charged with the re¬ 
sponsibility of securing linear standards, 
was informed, when visiting the NPL, of a 
gage-maker in Coventry who was making 
end standards of round design. The two 
Airy points were round and to size. The 
parallelism of the terminal surfaces thus 
could be determined by autocollimation; 
turning 180° reversed the error. Since the 
ends were symmetrical, better lapping for 
flatness was possible. 

It was decided to have this gage-maker 
make a series of gages of the highest quality 
without regard to cost. At the same time, 
a gage-maker in Germany did likewise. All 
gages were intercompared by the NBS, 
NPL and PTB. 

Several new series of standards were 
made. Between 1952 and 1961, Mr. Victory 
made 14 trips to NPL, the same number to 
the NBS and several to the PTB.* Each 
time he hand-carried the gages, working 
with the manufacturer, the NPL, the NBS, 
and the PTB to produce progressively bet¬ 
ter results. For instance, some of the first 
16 and 18 inch gages [master metric end 
standard is 480 mm] were within 50 mil¬ 
lionths of an inch [0.0013 mm] from nomi¬ 
nal. Determinations were close, because 
they were of excellent geometry, but it was 

*Extremely accurate checks were being made at 
an early date at the PTB with a new design 
Koester’s Interferometer—largely the work of Dr. 
E. Englehard. 


FIG. 200 —Master end standards are hand- 
carried to the major national bureaus of the 
world for intercomparison and certification. 
Albert Johnson, Chief Project Engineer at 
Moore, is responsible for standards. 


not felt safe to have such an error from 
nominal. Consequently, the gages were re¬ 
turned and new ones made. 

The end standards in use at Moore were 
30 millionths of an inch plus. With the pro¬ 
nouncement of the “International Inch” in 
1959 (page 108), it meant that the 18-inch* 
end standards would be 18 x 2 millionths 
of an inch, or 36 millionths of an inch 
longer, or a total of 66 millionths of an inch 
long. This was felt to be too great a gap to 
rely on the linearity of the electronic indi¬ 
cators then in use. The gages were returned 
to their makers to be re-worked to the new 
inch to less than 20 millionths of an inch 
from nominal. 

By this time, the No. 3 Universal Mea¬ 
suring Machine had been built and was in 
use. Through the use of the Measuring Ma¬ 
chine, a technique had been worked out to 
transfer the overall value of the end stan¬ 
dard to a Step Gage, the inch increments 
[30 mm increments with metric Step Gages] 
being established by sub-division (see pages 
180-185). 

Fig. 259 shows the result of a calibration 
of two of these Step Gages by the NBS and 
the NPL. The error from nominal is quite 
small and provides the desired 10 to 1 ratio 
to the Measuring Machine, guaranteed to 
an accuracy of 35 millionths of an inch 
[0.0009 mm]. 

However, the various national bureaus 
were reluctant to give an “accuracy of de¬ 
termination” closer than ± 15 millionths of 
an inch [0.00038 mm], because of the un¬ 
known metal and unspecified coefficient of 
expansion of the end standard to which the 
Nitralloy Step Gages were calibrated. This 
was something Moore did not think it could 
accept. As a consequence, Nitralloy end 
standards were made, hardened, ground, 
stabilized and brought to the gage-maker 

*The master metric length standard used is 
480 mm in length. This standard was unaffected 
by the change to the “International Inch’’; no 
metric figures are therefore listed. 
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pig. 261 —Calibration of end standards at 
the major bureaus is done on a continuous 
basis to assure a current record of trace¬ 
ability within each major nation. Close 
agreement by the bureaus gives greater 


confidence to the calibrations obtained. Note 
that the NPL, NBS, PTB and the HI PM 
agree on the length of End Standard 
No. 1-1?, within one ■millionth of an inch 
[ 0.000025 mm}. 


to be finish-lapped. This meant that the 
end standard, the Step Gage and the lead 
screw of the Measuring Machine were all 
of the same material, and thus had the same 
coefficient of expansion. 

Four sets of 12-inch and 18-inch [as well 
as 480 mm] end standards were made. The 
best of two of these gages were selected— 
the two closest to nominal having the best 
“geometry” as to micro-finish, flatness, and 
parallelism of the ends. 

After Mr. Victory’s death, the task of 
establishing standards fell to Mr. Albert 
Johnson, Chief Project Engineer, Fig. 260. 
Mr. Johnson was to hand carry these gages 
to the NPL, the PTB, BIPM and the NBS 
for calibration. 

Note that agreement is to 1 millionth of 
an inch [0.000025 mm] for gage No. 1-18, 
Fig. 261. This historical chart also provides 
one of the most accurate tests made on the 
dimensional stability of a material (Nitral- 
loy, in this case). Fig. 262 shows a Moore 
End Standard at the BIPM in preparation 
for measurement by interferometric means. 

These end standards calibrated by the 
bureaus are ultimately the authority for 
measurements of length performed with the 
Moore Measuring Machines. 

The reader may now appreciate that ac¬ 
curate linear measurement is synonymous 
with accurate temperature-control, and that 
the latter is deserving of as much attention 
as the former. Before proceeding, then, to 
linear measurement at Moore, it is necessary 
to delineate the “environment” where it is 
performed. 


TRACEABILITY OF MOORE STANDARDS 
Deviations from nominal are expressed in millionths of an inch. 


16 in. End Standard 


No. 216 

Bureau 

Calibration 

November 1958 

(Started) 

— 

June 1959 

NPL 

+ 1 

July 1959 

PTB 

+ 1.5 

August 1959 

NBS 

+ 3 

July 1960 

NBS 

+ 2 

March 1963 

NBS 

+ 2 

October 1964 

NBS 

-2 

18 in. End Standard 

No. 7587 

Bureau 

Calibration 

November 1958 

(Started) 

_ 

June 1959 

NPL 

+ 2 

July 1959 

PTB 

+ 10 

August 1959 

NBS 

+3 

July 1960 

NBS 

+ 6 

March 1963 

NBS 

-1 

October 1964 

NBS 

-2 

16 in. End Standard 

No. 2-16 

Bureau 

Calibration 

December 1962 

(Started) 

_ 

November 1963 

PTB 

-23.5 

March 1964 

NPL 

-26 

October 1964 

BIPM 

-22.1 

August 1965 

NBS 

-22 

Four Bureaus in agreement to 0.000004 (4 millionths) in., or 

1 part in 4 million. 


18 in. End Standard 


No. 1-18 

Bureau 

Calibration 

December 1962 

(Started) 

— 

November 1963 

PTB- 

-21.2 

March 1964 

NPL 

-21 

October 1964 

BIPM 

-22.1 

August 1965 

NBS 

-22 

Four Bureaus in agreement to 0.0000011 (1.1 
or 1 part in 16.4 million. 

millionth) in., 

480 mm End Standard 

No. 2-48CM (Metric) 

Bureau 

Calibration 

December 1962 

(Started) 

— 

November 1963 

PTB 

-32 

March 1964 

NPL 

-31.2 

October 1964 

BIPM 

-28.3 

August 1965 

NBS 

-31.1 


Four Bureaus in agreement to 0.0000037 (3.7 millionths) in., 
or 1 partin 5.1 million. 


NBS—U.S. National Bureau of Standards, Washington, D.C. 
NPL—National Physical Laboratory, Teddmgton, England 
PTB—Physikahsch-Technische Bundesanstalt, Braunschweig, 
WestGermany 

BIPM—Bureau International Des Poids et Mesures, Sevres, 
France 



fig. 202 —A Moore End Standard at the 
BIPM prior to measurement, by inter¬ 
ferometric means. 

Courtesy of 3!PM, 


167 









fig. 263 — The Moore Measuring Machine 
Laboratory is windowless, contributing to 
greater temperature uniformity. 




4. Temperature Control 

Moore’s temperature-controlled building, 
Fig. 263, is a two-story plus basement struc¬ 
ture, 40 ft. by 70 ft. [12.19 meters by 21.3 
meters], with a brick exterior. All the floors 
are of 12 in. [305 mm] reinforced concrete 
to give strength and stability. The base¬ 
ment and first floor are windowless. 

Within the structure are five separate 
temperature-controlled areas, with varying 
degrees of refinement, each designed to 
meet a special need. They are: 

Contract Inspection | 

Instrument Assembly 2 Below Ground 

Standards Room J 

Measuring Machine Laboratory—1st 
floor 

Jig Boring and Jig Grinding—2nd floor 

While the individual temperature- 
controlled areas are self-sufficient, they are 
integrated with one another as to power 
source and fresh air, Fig. 264. Moreover, 
they are connected with adjoining manu¬ 
facturing areas. 


MEASURING MACHINE 
LABORATORY (First Floor) 

The Moore Measuring Machine Labora¬ 
tory, Fig. 265, was originally designed and 
constructed expressly for assembly, inspec¬ 
tion and final corrections of Nos. 1R>, 3, 4 
and 5 Measuring Machines (the latter two 
are now made in a separate laboratory). This 
room, while accommodating many persons 
using lights, tools, hoists and inspection in¬ 
struments—all heat producers—still main¬ 
tains accurate temperature. From the be¬ 
ginning, the problem of how to control a 
room’s temperature to the closest possible 
tolerance and still have it a working room 
was uppermost. A temperature-controlled 
area is of little value if no one may use it 
or work in it. 
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EXPLODED VIEW OF MOORE’S 3-FLOOR 
TEMPERATURE-CONTROLLED BUILDING 


TOP FLOOR-TOOL ROOM, JIG BORING AND JIG GRINDING AREA 



MAIN FLOOR-MEASURING MACHINE LABORATORY 



- Temperature controlled air ducts 

3000 feet of copper tubing circulating 68“ f water in concrete floor 
Perforated ceiling 


Moisture proof insulated walls (8 layers) 



BASEMENT FLOOR-STANDARDS ROOM AND 
INSTRUMENT ASSEMBLY AREA 


Perforated ceiling in Standards Room 
Temperatuie control unit 


Incoming fresh air for both areas 

_ Freon pipes 

for Standards Room 
for Standards Room 
•mpcraiure controlled ait duos for Standards Room 
moisture proofed, insulated air ducts (under floor) 


FIG. 264 —-The Measuring Machine 
Laboratory Building is temperature- 

controlled with separate systems on each 
floor. Each floor contributes to the stability 
of the others. 
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fig. 205 — Universal Measuring Machines 
arc assembled and inspected in a 
temperature- controlled working Inborn tori/ 

(first floor). 
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fig, “Stratification" of room 
temperature is most difficult to prevent and 
is best achieved by the ‘‘reverse-flow" 
principle introduced by Moore. Note that 
the No, f Measuring Machine base has 
been insulated to guard against body heat 
■while being scraped. 


Problem of Stratification 
Because the Measuring Machines to be cali¬ 
brated stand at least 6 feet [2 meters] high, 
it was particularly important to eliminate 
“stratification” of room temperature, which 
could cause machine distortion and other 
inaccuracies, Fig. 266. The usual concept of 
a temperature-controlled room calls for rec¬ 
tified air to be introduced from the ceiling, 
spill out into the room, and be carried away 
either at the sides or near the floor, Fig. 
267. However, rectified air coming down 
into the room is at once met by the heat 
from lights and other rising heat currents. 
By introducing temperature-controlled air 
at the top, heat is prevented from being car¬ 
ried out and instead is brought back into the 
room. The rectified air is, therefore, not 
necessarily introduced at 68°F. [20° C], but 
at some lesser temperature. By the time air 
reaches work level, mixing within the room 
is depended on to create an average temper¬ 
ature of 68°F. [20° C]. 

When working with instruments, gages 
and smaller parts, a limited amount of strat¬ 
ification is permissible. The 68°F. [20° C] 
temperature is then held only within the 
narrow band at which measuring is per¬ 
formed, usually work-bench level. 

During Moore’s history of striving for 
ever-closer measuring tolerances of its loca¬ 
tional machines, the ability to control tem¬ 
perature had to progress hand-in-hand. 
From 1940, when its first temperature- 
controlled room was constructed, Moore 
has experimented with temperature control. 
There are now at least 20 separate systems 
placed strategically in the plant, varying in 
design and refinement. 

Considerable data was accumulated. It 
was noted that with the ceiling-to-floor air 
flow, while possible to control temperature to 
±Vio°F [less than 1 / 2 o°C] at one room level, 
it was virtually impossible to do so at all 
levels. Much effort was expended on the 
stratification problem. Maintenance of high 
static pressure greatly reduced the variance 
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FIG. 267 — In the usual conception of 
temperature-control, air is brought in at the 
ceiling and expelled at floor level. The 
disadvantages of this system are: 

1. Heal is brought back into the 
environment. 

2. The sensor ads slowly after averaging. 


8. Since the air cannot be brought in at 
68° F [20° C\, some of the cold air 
may sink to the floor, causing 
“stratification.” 

fig. 268 . The principal difficulty in an 
environment that is controlled by “reverse- 


flow" of air is to obtain sufficient “mix¬ 
ing”; however, this system has the following 
advantages: 

1. Heat is removed from the environment. 

2. The sensor ads immediately and 
must only correct incoming air. 

8. Air can be brought in at 68° F [20° C], 




FIG. 269 —Two separate temperature- 
control units maintain the temperature of 
the Measuring Machine Laboratory. Each 
unit consists of a heat and refrigeration 


cycle. These photos taken from mid-point 
of the room are combined to show one 
unit to the left and the second unit 
to the right. 
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of temperature from floor to ceiling. The 
conviction arose that the direction of air 
flow was the limiting factor to progress. 
One of Moore’s temperature-controlled 
rooms was then modified as a test area, 
where air was introduced at floor level and 
taken out at ceiling height. The results ex¬ 
ceeded expectations; not only was stratifi¬ 
cation cut by 80%, but temperature was 
maintained closer to ambient with the same 
control unit, Fig. 268. 

Design Features 

The laboratory itself was designed to in¬ 
corporate this new principle, as well as 
those other features culled from Moore’s 
experience, research, and study of other 
outstanding systems. Two independent 
temperature-control units, Fig. 269, were 
installed in an adjacent area, each consist¬ 
ing of a refrigeration and a heat cycle. The 
refrigeration cycle is believed to be a first- 
of-its-kind design. A variable speed com¬ 
pressor controls the “boil-off” rate of 
freon in a “flooded-coil” type of system. 
The “suction pressure” thus maintained 
has a direct relationship to the tempera¬ 
ture of the liquid freon. Instrumentation 
converts this suction-pressure to an elec¬ 
tric signal which, in turn, varies the speed 
of the compressor, Fig. 270. The refrigera¬ 
tion cycle thus is a self-contained tempera¬ 
ture-control system. Test-runs have shown 
it capable by itself of maintaining room 
temperature to ±34°F [ ± ¥%C approx.]. 

The heat cycle, consisting of heating coils 
which heat the air once past the cooling 
coils, is activated by temperature-sensitive 
probes. The heating coils are capable of 
pulsing on and off at the rate of 120 times 
per second, further reducing the limits of 
temperature deviation, Fig. 271. Each unit 
is zoned independently to control one-half 
of the room, but has more capacity than is 
needed to control the whole room by it¬ 
self. 

Through a “fail-safe” system of control 


vents in the feed ducts, either unit can be 
switched over to maintain the room in the 
event of a break-down or for repairs or 
adjustments of the other unit. The long 
“soak-out” period which would be neces¬ 
sary to bring the machines and gages back 
to equilibrium after an ambient tempera¬ 
ture change makes such a precaution eco¬ 
nomically justified. 

The sides of the room are actually a wall 
of individual ducts, 16 inches [406 mm] on 
center and individually adjustable, taking 
temperature-controlled air to the floor 
around the complete periphery of the room. 
The air is also fed down on four sides of 
three structural columns spaced apart in 
the middle of the room. It was found that 
this system did not provide as much “mix¬ 
ing” of air as hoped for and at a later date 
a series of vents were added to discharge 
rectified air over the work-benches. 

Temperature-controlled Floor 

68°F [20°C] water flows through over 3-2 
mile [800 m] of copper tubing in the 12- 
inch [305 mm] thick reinforced concrete 
floor. This network of piping, in four zones, 
contributes to stabilizing the temperature 
of the air at floor level, the most difficult 
area for eliminating stratification. 

The temperature-controlled rooms which 
sandwich the Measuring Machine Labora¬ 
tory further contribute to stabilizing its 
temperature. These rooms, together with 
the circulating water in the floor, also pre¬ 
vent the outdoor temperature, whether heat 
of summer or cold of winter, from creeping 
into the room via the steel reinforcing rods 
in the concrete. 



FIG. 270— The refrigeration cycle, unlike 
most systems, is capable of reacting 
independently to control temperature. 



FIG. 271^- The heating cycle acting on air 
which has already been closely-controlled by 
the cooling cycle. 
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fig. 27'$*— The Standards Room is located 
below-ground, and is a temperature- 
controlled room within a temperature- 
controlled room. Moore length standards 
are maintained here. 


STANDARDS ROOM 
(Below Ground) 

In the Standards Room, Fig. 272, are kept 
the following linear standards, the author¬ 
ity for length measurements performed on 
Moore Jig Borers, Jig Grinders and Mea¬ 
suring Machines: 

1. A series of master end standards, such 
as 9 in., 14 in., 16 in., 18 in., and 480 
mm, inter-compared by Moore and 
certified by the National Bureau of 
Standards, Washington, and the Na¬ 
tional Physical Laboratory, Tedding- 
ton, England, at regular intervals, 
Fig. 273. 

2. Moore Master Step Gages, which are 
a derivative of the end standards, but 
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FIG. 273 —Master end standards, which 
have been hand-carried to various bureaus 
of the world for calibration, are stored for 
safe-keeping in the Standards Room, 



calibrated in turn by the NBS, PTB, 
BIPM and the NPL, Fig. 274. These 
gages are accurate to a few millionths 
of an inch [tenth of a /im]. In this room 
is performed the transferal of accuracy 
from master standards to Step Gages 
by measurement, calibration and cor¬ 
rection. 

Importance of 68° F [20° C] 
Temperature 

Consider that an 18 inch [457 mm] Step 
Gage whose accuracy must be in the realm 
of 2 millionths of an inch [0.00005 mm] will 
be in error by 2 millionths of an inch 
[0.00005 mm], when its temperature differs 
by as little as 1/50°F [1/90° C] from that of 
the master standard to which it is being 
compared. Then it will be appreciated that 
the very highest order of temperature con¬ 
trol system must be used in the Standards 
Room. 

A location below ground level was pre¬ 
ferred for the Standards Room because of 

(1) the naturally low level of vibration and 

(2) the fact that ground temperature would 
contribute to the stability of the room. 
While the location below-ground was ad¬ 
vantageous, it was recognized that the walls 
and floor of the room, which had to be held 
to exactly 68°F [20° C], could actually be 
exposed to ground temperature, normally 
55°F [13°C], 

C onsequently, it was decided t o make the 
Standards Room a “room within a room.” 
The space surrounding it would become a 
new area having its own temperature con¬ 
trol system and would encompass instru¬ 
ment assembly, Fig. 275, rotary table cali¬ 
bration, Fig. 276, and contract inspection, 
Fig. 277. For this area, the requirements 
would be less stringent, but would contrib¬ 
ute to stabilizing the temperature of the 
Standards Room. Also, the wood floor of the 
Standards Room would be elevated from 
the concrete, and the space between insu¬ 
lated, moisture-proofed, and utilized to 


fig. 274 —Moore Master Step Gages, whose 
lengths are derived from master end 
standards, are lapped to final accuracy, 
calibrated and stored in the Standards 
Room. 
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fig. 275 —The Rotary Table Assembly 
Room adjoins the Standards Room on one 
side. 'The Standards Room can be seen 
through the windows. 



fig. 270 —Adjoining the Standards Room is 
the rotary table lapping and calibration 
area. 
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Flo. 277 — The Contract Inspection Room, 
located adjacent to one end of the Standards 
Room, specializes in unusual inspection 
tasks performed with Universal Measuring 
M achines. All the instruments, gages and 
standards at Moore are at the disposal 
of its personnel. 

























FIG, 270 --The success of the Moore 

reverse-flow air principle encouraged its 
adoption by other leading laboratories, such 

as the Instituto Galileo. 

Courtesy Prof. Raffaele Ciambrone, Instituto Galileo 
Galilei, Milano, Italy. 


conduct the rectified air to the opposite 
wall. These precautions isolate the Stan¬ 
dards Room from external temperature 
conditions. 

68° F [20° C ] is controlled to the highest 
order, Fig. 278. The success of the reverse- 
flow principle in the Measuring Machine 
Laboratory, especially in minimizing strati¬ 
fication, encouraged the adoption of a simi¬ 
lar system in the Standards Room. In the 
transferal of accuracy, master gages are set 
up on the table of a Moore Measuring Ma¬ 
chine, the latter being employed only for 
straight-line movement. If the room were to 
be subjected to stratification, the tempera¬ 
ture of the lower strata of the room would 
tend to be conducted through the machine, 
influencing the gages. By minimizing strat¬ 
ification, the mass of the machine then acts 
as a stabilizing “heat sink,” assuring that 
the gages being compared are as close as 
possible to 68°F [20°C], and even more im¬ 
portant, of identical temperature. 

In conclusion, three main design features 
assure the Standards Room being controlled 
at 68°F [20°C] to the highest order: 

1. The reverse air-flow system; 

2. The exclusion of heat-producing ele¬ 
ments; 

3. The greater length-to-width ratio of 
the room enables the rectified air in¬ 
troduced at floor level to be more 
completely infiltrated in the room. 

The inherent logic of the “reverse-flow” 
system has influenced the adaption of this 
system in other leading standards labora¬ 
tories, such as the Instituto Galileo Galilei, 
Milan, Fig. 279.* 



*Professor Raffaele Ciambrone informed the au- 
thor at the 1969 Microtecnic Exposition, Zurich, 
that it was operating to complete satisfaction. 
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fig. 280— Illustrated is the manner in 
which the step gage is isolated from 
external sources of temperature-differential 
during calibration. 


5. Transferal of Accuracy* 

TEMPERATURE PRECAUTIONS 


Standards room 
controlled 
environment 
(reverse flow) 



Plexiglass cage around 
measuring machine. 

Plexiglass cover 

Step gage rests on 
measuring machine. 


Separately controlled environment 


-14 ft below grade - 



Fig. 280 shows the location of the Step Gage 
during transferal of accuracy from end 
standard to Step Gage. The floor is 14 feet 
[3.6 meters] below grade on “hardpan.” 

The Instrument Assembly Area and the 
Inspection Center Area surrounding the 
Standards Room are separately tempera¬ 
ture-controlled. A plexiglass cage surrounds 
the machine. The end standard and Step 
Gage to be compared both rest on the Mea¬ 
suring Machine table. The gages them¬ 
selves are “buried” behind a plexiglass 
cover, Fig. 281, leaving only a small slot to 
permit indicator movement. Aluminum foil, 
used as a reflective shield over the cover, was 
removed in the picture to show the setup. 

Thus isolated, the Step Gage assumes 
table temperature fairly quickly. The end 
standard, having less surface contact, 
requires more time to reach ambient. This 
potential source of a temperature differ¬ 
ential is minimized by placing the end 
standard in contact with the Step Gage. 

Although the temperature of the air 
about the length standards is probably not 
held closer than l/50th of a degree Fahr¬ 
enheit [1/90° C, approx.] from ambient, 
with the above temperature precautions, 
the gages, once “soaked out,” are virtually 
identical in temperature. However, as an 
added precaution, and for convenience, it is 
useful to be able to probe for temperature 
differentials with thermocouples anywhere 
within the immediate gaging environment, 
Fig. 282. Each thermocouple can be made 
to register on a galvanometer inside a “black 
box” located in the Standards Room, Fig. 
283. The thermocouple can also be com¬ 
pared to a certified master thermometer 
which is buried in a copper block inside the 
black box. 

*Most metric equivalents are omitted from this 
section by intention for the sake of clarity, since the 
same procedure described is also followed for metric 
gages. 


fig. 281 —The Step Gage is compared to 
the end standard while both rest on the 
table of the Universal Measuring Machine. 
Slots in the plexiglass cover are only wide 
entuyh to facilitate indicator mtvement. 


Aluminum foil, which in practice is 
placed over the cover and used to reflect 
radiant heat, was removed for the 
photograph. 

Photograph courtesy American Machinist. 
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fig. 282 — Thermocouple s are used to probe 
the Step) Gage along its length and also 
other areas of the gaging set-up in order to 
delect any temperature differentials. 



«ASU«!NC 

MACHINE 



fig. 283 —Each thermocouple in the gaging 
area can register temperature through 
switching by means of a galvanometer inside 
a ‘‘black box.” Thermocouples can also be 
compared to a calibrated master thermom¬ 
eter buried in a copper block inside the 
‘‘black box.” 


















fig. iSi—The ultimate standards of length 
at Moore are end standards of various 
lengths. The purpose of the “wrung-on 
piece" is to act as a reflector in the optical 


arrangement when compared to the krypton 
86 standard. The same wrung-on piece is 
also used in “transferal" of accuracy from 
end standard to Step Gage. 



METHOD OF TRANSFERAL 

For the transferal of accuracy, two elec¬ 
tronic indicator gage heads are fastened to 
a steel bar, which is, in turn, fixed to the 
Measuring Machine spindle housing. A se¬ 
ries of tapped holes are provided in the bar 
to enable separation of the gage heads in 
nominal 1 inch increments up to 18 inches 
and 30 mm increments up to 480 mm. 

The electronic indicator used is of the 
differential-type, reading to 1 millionth of 
an inch [0.000025 mm]. The meter shows 
the difference between the length of the 
18-inch End Standard, and the 0-18 inch 
step of the Step Gage. 

To compare the two gages, the 0-18 inch 
tapped holes of the support bar are used, 
and a “zero” is set against one end of the 
end standard and the lapped wrung-on 
piece at its other end (the wrung-on piece 
was used previously as an optical mirror 
for light-wave calibration at the bureaus). 
The purpose of the wrung-on piece in this 
case is to have both indicators “face the 
same direction” as the pads on the Step 
Gage, Fig. 284. 

Only a short movement of the Measuring 
Machine cross-axis is needed to bring both 
indicators into contact with the 0 to 18 inch 
pads of the Step Gage. The difference in 
length is noted. 

Next, the Step Gage is removed for lap¬ 
ping either its zero or 18 inch pad, depend¬ 
ing on whether long or short, Fig. 285, 
taking into account the calibration of the 
end standard. This sequence is repeated 
many times. Once the 18-inch step is es¬ 
tablished, the 16-inch End Standard is used 
to secure the 16th inch step in a like manner. 

The transferal of the 16 and 18 inch 
lengths requires three or four months of 
elapsed time due to the time required for 
the Step Gages to return to ambient each 
time after being lapped and handled, and 
to make the many necessary cross-checks. 


fig. 285 —Gaging faces of the Step Gage are 
lapped on a special fixture. Great care 
must be taken in the lapping operation to 
maintain the “geometry" of the faces. If 


one step is reduced only a few millionths of 
an inch [tenth of a pm] more than 
intended, the only means of correction is to 
reduce all the other steps a like amount. 
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SUBDIVIDING THE STEP GAGE 


Once the 16 and 18 inch steps are secured, 
the remaining steps can be established by 
subdivision. Absolute temperature is not as 
important as is uniformity of temperature 
throughout the length of the Step Gage. 

Using a different set of tapped holes in 
the support bar, 0 to 9 inch and 9 to 18 inch 
steps are compared by moving the longitu¬ 
dinal axis of the Measuring Machine 9 in¬ 
ches. By lapping the 9 inch step until half¬ 
way between the 0 and 18th inch steps, it 
thereby becomes exactly 9 inches. The in¬ 
spection is sensitive: If the 9-inch step is 
not midway between 0-18 inch, it shows as 
a double indicator error. 

The advantage of the 16-inch End Stan¬ 
dard is now apparent: 16 inches being divisi¬ 
ble to obtain 8 inches, 8 inches to get 4 
inches, etc. This classical method of sub¬ 
division is preferred since rather than accu¬ 
mulate errors, as is inherent where several 
smaller blocks are added together, the steps 
theoretically become closer to “truth.” 

Many cross-checks are made. For in¬ 
stance, all the three-inch steps should now 
agree. The 16th and 18th steps are estab¬ 
lished by comparison to the 16-inch stan¬ 
dard and 18-inch standard, respectively. 
Since the 17th inch step is secured by sub¬ 
division, the increments 16-17 inch and 17- 
18 inch should both be exactly one inch. 

Total elapsed time for creating a Master 
Step Gage is 9-12 months. 

STEP GAGES INSPECTED 
BY THE BUREAUS 

The Master Step Gage is next calibrated at 
the national bureaus. After once again de¬ 
termining the length of the end standard 
by light-wave measurement, the Step Gage 
is calibrated by the same method of sub¬ 
division, again using a Moore Measuring 
Machine. 








fig. 287 —The design of the Step Gage 
enables t inch [SO mm in metric ] “steps” 
to be contained in a single bar. 

Because it is an “end-measure,” great 
accuracy can be achieved. 




SECOND GENERATION 
STEP GAGES 

The Master Step Gage, after calibration by 
the national bureaus, is used for the crea¬ 
tion of Second Generation Step Gages. 
While both rest on the Measuring Machine 
table, the Step Gage to be lapped is com¬ 
pared against the Master Step Gage inch 
by inch over its whole length, Fig. 286. Ten 
millionths of an inch [0.00025 mm] accuracy 
is achieved relatively quickly. The greatest 
amount of time is consumed in bringing a 
Step Gage from this stage to about 1 or 2 
millionths of an inch [0.000025 to 0.00005 
mm] accuracy. 

It must be remembered that correcting a 
Step Gage is unlike the problem of correct¬ 
ing a standard made up of many small one- 
inch [25 mm, for example] blocks. If one of 
the pads of the Step Gage is lapped 2 mil¬ 
lionths of an inch short [0.00005 mm], for 
instance, the only means of correction is to 
reduce all the other steps a like amount. 

The procedure in transferal, subdividing 
and lapping metric step gages is similar to 
that employed with inch step gages, except 
that the master end standard is 480 mm and 
the increments of the Step Gage are 30 mm. 

MOORE STEP GAGE 
Design Considerations 
The design of the Moore Step Gage, Fig. 
287, allows end standard accuracy to be ob¬ 
tained, while avoiding the inaccuracies 
which can result from fastening a number of 
gages together. One inch [30 mm in metric] 
“steps” are contained in a single bar. 

Step Gages are made in a number of 
lengths, 12 in., 16 in., 18 in., 24 in., and 480 
mm corresponding to the travel of Moore 
machines. Note in Fig. 287 again that the 
gaging surfaces are as close as possible to 
the neutral bending plane, commensurate 
with rigidity of the bar. Both sides and also 
the bottom of the Step Gage are lapped 
straight, parallel andsquareto 10 millionths 


FIG. 288 — Because the gaging surfaces of 
the Step Gage are not on its neutral axis, 
when it rests on a surface that is not flat, 
its effective length is altered through 
bending. If the surface on which it is 
supported is convex (as illustrated), 
effective length is increased; if concave, 


effective length is decreased. The additional 
effort required to establish true surfaces 
(rather than the use of “Airy points”) is 
rewarded by greater control of temperature 
of the Step Gage. 



of an inch [0.00025 mm]. The gaging pads 
are lapped square to the sides and bottom. 

Step Gage designed to be used flat. During 
calibration, the Step Gage is supported its 
full length on the Measuring Machine table 
to gain maximum surface contact. The 
Measuring Machine, a large mass at ambi¬ 
ent and slow to change, acts as a “heat 
sink,” and lends stability to the standards 
in contact with it. For this additional rea¬ 
son, the Step Gage is given a low profile to 
avoid being influenced by air temperature. 

Two Step Gages, compared while resting 
on the Measuring Machine, will automati¬ 
cally be closer in temperature than if either 
or both are supported on Airy points. Simi¬ 
larly, when the Step Gage is itself used to 
calibrate the measuring elements of a ma¬ 
chine tool, it will more likely assume ma¬ 
chine temperature, rather than air tempera¬ 
ture. This manner of support requires a 
perfectly flat surface, since bending of the 
Step Gage alters its length, Fig. 288. The 
requirement of a flat support surface would 
at first seem a needless inconvenience when 
Airy points support could more simply be 
used, until one considers that the additional 
effort is compensated for by much more 
control in the temperature of the Step Gage. 

6. Choice of Length- 
Measuring Element for 
Moore Measuring 
Machine 

The positioning accuracy of Moore Jig Bor¬ 
ers and Jig Grinders up to the early 1950’s 
was in the order of one or two ten thou¬ 
sandths of an inch [3 to 5 jim]. The goal, 
however, was for machine positioning with¬ 
in 35 millionths of an inch [0.0009 mm]. At 
this juncture in the company’s history, 
there was naturally some reflection as to 
what sort of measuring system could be 
used to meet this accuracy. 

The measuring elements used in the most 


accurate coordinate-locating machine tools 
are many and varied: the standard preci¬ 
sion scale, Moire-fringe type of scale, end 
rods, lapped pins in a trough read against 
an indicator, lead screws, ball screws and 
many more. In general, though, they fall 
into one of three categories: scale, end 
measure and lead screw. 

END MEASURE 

The end measure system, however excel¬ 
lent when used purely as a length standard, 
loses much of its desirability when used 
with a machine tool. In all its forms, the 
end measure system is slow and cumber¬ 
some; since the rods must be accessible, 
they are exposed to air temperature and to 
heat from handling, as well as to dirt and 
dust. 

Positioning accuracy depends on how 
well the several bars that make up each 
measurement are matched end-to-end. The 
smallest increments of movement require a 
micrometer and an indicator, a further divi¬ 
sion of responsibility for positioning. 

PRECISION SCALE 

The advantages attributed to the precision 
scale, see Fig. 258, are twofold: first, that a 
single standard can be divided into many 
small increments; second, since used only 
as a reference for positioning (unlike a lead 
screw which both positions and measures), 
there is no build-up of heat in the scale 
from movement or wear from use. 

With large machines requiring fast move¬ 
ment for many varied positions, the preci¬ 
sion scale has much in its favor. However, 
as noted earlier (pages 165-166), repeat¬ 
ability is only 20 to 30 millionths of an inch 
[0.5 to 0.7 jum]. Errors from nominal are as 
much as 80 millionths of an inch [2 yum]; a 
calibration chart would have to be used. 
Since the lines, once ruled, cannot be al¬ 
tered, no adjustment can be made to match 
the scale to the minute individual charac¬ 
teristics of each machine. Finally, a 


micrometer screw still must be used to 
position to thousandths and ten thou¬ 
sandths of an inch [hundredths and 
thousandths of a millimeter] — an addition¬ 
al source of error. 

Even supposing a photoelectric pick-up 
to be used for the even 50 thousandths of 
an inch line divisions [1 mm when metric], 
the increments within 50 thousandths of an 
inch [1 mm] are given no increased accur¬ 
acy. 

LEAD SCREW 

The lead screw has many inherent advan¬ 
tages—high magnification of setting (as 
much as 100 or 200 to 1), “stepless” mea¬ 
surement; single, undivided authority for 
positioning and measuring; ease and speed 
of setting. If properly constructed and as¬ 
sembled, repeatability can be to a few mil¬ 
lionths of an inch [tenth of a yum]. 

Problem of Wear. A primary considera¬ 
tion with regard to the lead screw is its 
wear in use. Often, when the wear of a lead 
screw is mentioned, what is actually being 
considered is an unhardened lead screw, 
whose thread has been chased in a precision 
lathe. Compensation for initial lead error, 
or from wear in use, is through a follower 
cam and linkage which advances and retards 
either the nut or the vernier-dial. Common 
practice is to refile this cam yearly. 

Moore had never considered this sound 
machine tool design. Soft screws give no 
permanence of accuracy. Cams or link¬ 
ages are likely to introduce as much error 
as they were intended to remove. It is un¬ 
likely that periodic errors (errors within one 
turn of the screw) can be compensated for 
in this manner. 

Problem of Heat. Another disadvantage to 
the lead screw sometimes cited is the pos¬ 
sibility of its warming up from over-zealous 
use, causing a slight linear increase in its 
length. However, considering the precision 
scale as an alternative, errors due to warm¬ 
up of the lead screw in its full travel would 
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fig, 289 —For specialized purposes, and 
when conditions are rigidly controlled, 
exceptional accuracy may be attained by 

employing a laser interferometer to measure 


displacement. This ruling engine for 
diffraction-gratings was built in cooperation 
with Professor George Stroke of the State 
U niversity of New York, Stony Brook. 



be no more serious than the repeatability of 
the scale at one position. In addition, the 
light required to read the scale is a source 
of heat, so must be immediately turned off 
once settings are made, sometimes accom¬ 
plished automatically with a time-delay 
switch. 

Thus, regardless of the linear system used, 
the problem of heat is not entirely absent. 


LASER INTERFEROMETER 

Actually a fourth system of measurement 
should be mentioned, that of light waves, or 
fringes. Only recently, the laser interferom¬ 
eter has been used, not only to calibrate ma¬ 
chine tools, but to monitor or measure their 
movement. Some foresee this as the ulti¬ 
mate for machine tool positioning. It is not 
effected by wear, nor heat from movement, 
and works directly with an unalterable 
standard. 

The laser interferometer has already been 
applied to the Moore Measuring Machines 
built for Dean George Harrison of M.I.T. 
and Professor George Stroke of the State 
University of New York, Stony Brook, for 
use in ruling engines, Fig. 289, and for ad¬ 
ditional highly specialized purposes. Under 
specific, controlled conditions, this may 
work quite well. It should be emphasized 
again, however, (see pages 121-122) that 
using light-wave measurement, tempera¬ 
ture control must be superb or the tem¬ 
perature of the piece to be machined or 
measured must be exactly determined so 
that precise compensation can be made to 
the extent the piece varies from 68°F. 
[20°C.], taking into account the coefficient 
of expansion of the piece. When the work- 
piece under consideration has nearly zero 
coefficient, as in ruling glass or quartz, the 
advantage is then with light-wave measure¬ 
ment, which theoretically can be con¬ 
sidered to have “zero” coefficient of 
expansion. 

Conclusion —Regardless of the measuring 
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fig. 290 — The accurate performance of the 
lead screw is dependent on the rigid 
adherence to many exacting design 
specifications. 



Slightly scaled down view illustrates the design, finish, and fit for the Moore No. 3 Jig 
Borer and Jig Grinder. The Measuring Machine has the same lead screw construction, 
but is brought to closei tolerance and, with a larger dial, reads direct to 20 millionths. 


MOORE MASTER LEAD SCREW ASSEMBLY 


Graduation = 150to 1 magnification 
One revolution of 15" circumference dial 
advances table .100” (10 pitch screw) 
.001" = .150" on dials 
.0001" = .015" on dial 
Can be set to ± 10 millionths with naked eye 


2" long bearings (7" total spread) 
produce perfect alignment- 


No. 3 Measuring Machine tolerance 
is millionths overall 


Ring and Plug GageTolerances 
for Positive Line-Up 
All guide sleeve holes hardened, ground 

and lapped alike to.+25 millionths—. 

— 0 millionths 


Ail Nitralloy lead screw bearings hardened, 

ground, and lapped alike to.+ 0 millionths —j 

—25 millionths 


Nut-handscrapedforexact line up 


Five millionths tolerance thrust 
balls ride on thrust surfaces 
lapped to flatness and squareness 
of five millionths... toeliminate 
friction and camming of lead screw 


Oil chamber piped from 1-shot reservoir 


Lead screw pilot 
bearing; line bored 
and line reamed 
to .0001" tolerance 
alignment with 
sleeve bearing, 
.0001" clearance 


21" Long Lead Screw ■ 
Short Lead Screw- 


Result: Clearance (Minimum space 
required for oil film). 


Max. 150 millionths 
Min. 100 millionths 
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element used, repeatability is still depen¬ 
dent on sensitivity and the ability to control 
fine increments of movement—for which a 
hardened, ground, and lapped lead screw, 
properly supported, can hardly be sur¬ 
passed. This being so, it would seem logical 
to put the additional effort required into 
making the lead screw accurate as well. 

Moore finally decided that the accuracy 
sought could only be achieved, not through 
an alternate system, but by the principle of 
the hardened, ground and lapped lead 
screw which had been pioneered as early as 
19 31. The M aster Lead Screw which evolved 
is explained in the following section. 

7. Moore Master Lead 
Screw 

The Moore Master Lead Screw, assembled 
in the Measuring Machine, Fig. 290,* repre- 

*“Moore Master Lead Screw,” Moore Special 
Tool Company, Inc., 1960. 


sents the culmination of the efforts to es¬ 
tablish the most accurate measuring ele¬ 
ment possible, at once traceable to the sev¬ 
eral major bureaus of standards. The lead 
screw has a 0.100 inch lead in the inch sys¬ 
tem and a 3 mm lead in metric machines. 

MATERIAL OF 
THE LEAD SCREW 

Moore Master Lead Screws are made of a 
nitrided steel—“Nitralloy” (see page 121). 
Characteristics of Nitralloy: A surface 


hardness on the Rockwell “C” Scale of 
approximately 75, or 94 on the 15N super¬ 
ficial hardness scale, or 84 in the 30N 
scale. The case depth is only 0.020 inch 
[0.5 mm], supported by a Rockwell “C” 
23-32 tough core. 

The hardened case approaches carbide in 
wear-resistance, yet its coefficient of expan¬ 
sion, 0.0000066 inch per inch per degree 
Fahrenheit [0.000012 mm/mm/°C], agrees 
with that of most steels. 

During the hardening cycle, the lead 



passes 


Inches of travel 


FIG. 291. —Chart shows the initial calibration 
of a lead screw, and then its calibration 
after 10,000,000 passes on only a iws-inch 
[50.8 mm] section of its length. This 
severe lest is estimated to be equivalent to 
800 years of normal use. 
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fig. 292 --A tremendous investment in 
supportintj equipment is necessary to 
produce accurate lead screws, such as the 
filtered temperature-controlled oil that 
floods the lead screw while being 
thread-ground. 
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screw is hung vertically in a furnace having 
a gaseous nitrogen atmosphere. A thermo¬ 
chemical diffusion process takes place. It 
requires 48 hours at 975°F [524°C] in the 
furnacefornitridingpenetration to a depth 
of 0.012 of an inch [0.305 mm), after which 
it is allowed to cool slowly. Additional hours 
in the furnace do not increase hardening 
depth appreciably. 

After nitriding, the lead screw is stress- 
relieved in raw ammonia dissociated, ap¬ 
proximately 50% at 1000° F. [538° C] for 
two hours at heat. 

Nitralloy has only minimum distortion in 
the hardening cycle and is thereafter of ex¬ 
ceptionally high dimensional stability. This 
can be attributed not only to its metallurgi¬ 
cal composition, but also to the low harden¬ 
ing temperature and the absence of a 
quenching “shock.” 

Innumerable Nitralloy end standards 
and Step Gages and thousands of lead 
screws have already been manufactured at 
Moore. With the ability to measure as ac¬ 
curately as possible, and on the basis of 
very careful records, it must be concluded 
that Nitralloy—when properly machined 
and properly heat-treated—does not mea¬ 
surably change in length over many years. 
The history of Nitralloy end standards 
maintained by Moore and calibrated at the 
national bureaus is also another proof of 
Nitralloy’s stability. In addition, Nitralloy 
is virtually immune to wear, as evidenced 
by the chart of a lead screw calibrated 
after undergoing the equivalent of 800 years 
of use, Fig. 291.* 

Nitralloy thus possesses three prime req¬ 
uisites to be the material for a machine’s 
measuring element: 

1. Coefficient of expansion equivalent to 
most steels; 

2. Immunity to wear; 

3. Dimensional stability. 


*“150 Year Old Lead Screw,” Moore Special 
Tool Company, Inc., 1965. 


fig. 293— -Diagrammatic illustration of Hie 
system used to control the temperature of 
the lead screw while being thread-ground. 
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fig. 294 —When being thread ground , the 
lead screw is supported carefully to assure 
concentricity of support to thread, and 
flooded with temperature-controlled oil 
to maintain exact lead. 


THREAD-GRINDING OF 
THE LEAD SCREW 

All outside diameters of the lead screw are 
ground in the soft state, the thread being 
ground from the solid. After the hardening 
and stabilizing cycle, only a few thou¬ 
sandths of an inch [several hundredths of 
a millimeter) are reground off the thread to 
establish lead. 

The nitrided “skin” decreasesin hardness 
roughly proportional to the amount of ma¬ 
terial removed in grinding. To maintain 
maximum hardness, then, the lead is de¬ 
liberately ground in error a calculated 
amount in the soft state to allow for change 
in length in hardening. The thread is care¬ 
fully picked up after hardening to balance 
out stock removal. 

In attempting to thread-grind relatively 
long pieces to any degree of accuracy, there 
are many thermal variables to contend 
with: changing room temperature; heat 
from motors, from gear boxes, from the 
grinding head or from cutting. All diminish 
the accuracy that is sought quite apart from 
the errors inherent in the thread grinder. 

Controlling air temperature is essential, 
but only a beginning. Much experimenta¬ 
tion was conducted to generate accurate 
leads in thread-grinding. Three such ex¬ 
periments were: grinding the screw long in 
an attempt to compensate for heat de¬ 
veloped during grinding; operating at 
artificially elevated temperatures, and 
manipulating lead through gearing. Some 
of these methods produced modest success. 

It was finally resolved, however, that the 
only true solution would be to flood the lead 
screw with a cutting oil of a closely controlled 
temperature during the thread-grinding op¬ 
eration. A major investment was made to 
accomplish this. The advantage of this 
method is that the temperature of the lead 
screw is now dominated by the cutting fluid. 
Further, with the increasing skills which 
developed at Moore with temperature- 
control systems, it was found that very 
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FIG. 296 —Overall lead and periodic error 
are inspected in a special lead-checker. 
After the nut is filled and the lead screw is 
assembled in the Measuring Machine, the 
lead screw undergoes final calibration and 
correction. 


FIG. 295 — After thread-grinding the lead 
■screw, thread drunkenness and lead error 
are corrected by hand-lapping. 


fig. 297 — Two charts of a lead screw made 
on the lead-checker arc shown. Left chart 
was made of a lead screw after thread 
grinding. Right chart was made after 
hand-lavving. 


190 














fig. 298—The lead screw thread is lapped of laps in gradual increasing pitch- 

many times until correct lead is established, diameters is used to mate the- nut to the 

so that- actual pitch diameter may vary lead screw. Once mated, the nut slays with 

from one lead screw to the next. A series the lead screw ‘‘for life,” 


sensitive and immediate corrections in lead 
could be effected through slight adjust¬ 
ments in the temperature of the cutting oil. 
See the photo of the plumbing system, Fig. 
292, and the diagram of the oil-control 
system, Fig. 293. 

To be able to grind a lead screw to 
within 0.0002 inch accuracy [0.005 mm], 
many additional requirements must be met. 
Most importantly, dressing diamonds must 
be replaced often to maintain free-cutting 
wheels; grinding wheel and drive motor 
must be dynamically balanced. 

To ensure concentricity of thread to 
journal, the lead screw rotates on its own 
lapped journal bearing, supported on a 
lapped carbide bushing while the lead is 
being ground, Fig. 294. Thus supported, the 
centers are not critically related to the accu¬ 
racy of the thread. However, the spindle 
must run true and be free of camming. 

LAPPING OF THE THREAD 

Following thread-grinding to 0.0002 inch 
accuracy [0.005 mm], the thread is hand- 
lapped to further correct lead. The cast- 
iron split lap used, Fig. 295, has a useful 
life sufficient to correct only one lead screw. 
A Moore-designed Lead-Checker, Fig. 
296, provides a continuous record of overall 
lead error, including periodic error. Fig. 297 
shows a comparison of ground and lapped 
lead screws. The Moore method of lapping 
corrects lead while maintaining uniform 
pitch diameter. 

LEAD SCREW NUT 

The material used for the lead screw nut is 
an aluminum bronze alloy, which is ex¬ 
tremely resistant to wear. 

The procedure in threading the nuts is as 
follows: First establish an accurate bore to 
fit the pilot of the first tap. Follow by a se¬ 
ries of taps, each with the same lead as the 
screw, but having slightly greater pitch 
diameters. Gradually increase the size of 
the taps until the nut has the same nominal 




fig. 299 —The nut has been lapped until it 
can be just forced on the lead screw. 
Following this , the nut is burnished to the 
lead screw using only light oil to achieve a 
perfect fit. 
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fig. 300-“ Approximately 1000 lead screws 
are produced at Moore each year. After 
lead is corrected, the nut is fitted and 
remains with the lead screw (far right), 
later to be part of the lead screw assembly. 
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fig. 301 — Highly specialized equipment 
used to align the lead screw precisely in 
the base of the Measuring Machine. 


pitch diameter as the screw, and thread onto 
the lead screw with a tight fit, Fig. 298. 
The nut is run back and forth on the lead 
screw for hours using light oil only while 
the screw is supported on centers in a 
lathe. This continues until the nut is 
burnished into perfect thread flank con¬ 
tact with its mating lead screw, Fig. 299. 

As a final test of fit, the lead screw and 
nut together are rotated slowly on centers. 
The nut body is unsymmetrical; that is, it 
has a flange which is used for mounting to 
the machine. If too tight a fit, the nut will 
rotate with the lead screw; if too loose, the 
nut will hang with its mounting flange 
straight down. 

A perfect fit is attained when the nut 
flange is maintained within a prescribed 
position while run the whole length of the 
lead screw; that is, a balance between grav¬ 
ity pulling it straight down and friction 
tending to cause it to rotate with the screw. 
Once fitted, the nut remains with the lead 
screw “for life,” Fig. 300. 

LINEUP OF THE LEAD SCREW 

In Fig. 301 is shown some of the specialized 
equipment used to align the lead screw. A 
hardened, ground and lapped steel test bar 
is supported at the dial-end by a lapped 
steel bushing and further down its length by 
a lapped carbide bushing. The carbide 
bushing is adjusted until the test bar, when 
indicated, is exactly parallel to the ways. 
The test bar is replaced with identical bars 
that have cutting tools to bore and ream the 
end journal from the same support, Fig. 302. 

After boring, the alignment is inspected 
with the test bar in a manner similar to that 
shown in Fig. 302, except that instead of 
being supported in the carbide bushing, the 
test bar is supported in the end journal just 
bored. This inspection can be seen at a 
later stage in Fig. 323. 

The bottom flange of the lead screw nut 
is hand-scraped to align its axis with these 
journals, Fig. 303. With the close fits speci- 
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fig. 802 —The test bur is first indicated 
parallel in two directions to the machine 
axis using an adjustable, lapped carbide 
bushing. Once the lest bar is aligned, the 


bushing is locked in place. The test bar is 
followed by identical bars that hold cutting 
to*ls used to bore and ream the end journal 
in-line. 


fig. 303— The mounting surface •/ the fig. 30i—The final test for line-up of the 

lead screw nut is hand-scraped to align its lead screw is severe but simple. A slightly 
axis with that of the lead screw. eccentric “dog” must fall free of its own 

weight on either side of center. With the 
close fits specified, only ci slight 
misalignment would cause the lead, screw 
to bind. 
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fig, 305' --In the final Step Gape calibration 
of the Measuring Machine for positioning 
accuracy, actual displacement is measured. 
The lead screw is disassembled only a few 
times at this stage for careful corrective 
lappings o f lead. 


fled (see Fig. 290), the lead screw will 
“bind” in rotation with only the slightest 
misalignment. 

The test for line-up is severe, but simple: 
a slightly eccentric “dog” must fall free of 
its own weight when released from either 
side of top-dead-center, Fig. 304. With the 
close fits specified, a 0.0001 in. misalignment 
[0.0025 mm] would cause the lead screw to 
bind. This operation is very critical, entail¬ 
ing partly an engineered solution and partly 
a high degree of craftsmanship. 

STEP GAGE CALIBRATION 
OF LEAD SCREW 

It will be remembered that the lead screw 
had been lapped to approximately 0.0001 
inch [0.0025 mm] accuracy using the lead 
checker. For final calibration, the lead 
screw is assembled in the Measuring Ma¬ 
chine, and actual table displacement is mea¬ 
sured, using the Moore Step Gage, Fig. 
305. 

Advantages of the Step Gage method 
include: 

1. Allows use of high magnification elec¬ 
tronic indicators; 

2. Lead screw can be compared to Step 
Gages of 2-3 millionths of an inch ac¬ 
curacy [0.00005 to 0.00007 mm]; 

3. Only a few careful lapping operations 
are necessary so that the fit of nut to 
lead screw is maintained; 

4. Calibration is made under the identi¬ 
cal conditions in which the lead screw 
is to be used, allowing corrections for 
individual characteristics of each ma¬ 
chine. 

Calibration of the lead screw takes place 
in a 68°F [20° C] temperature-controlled 
environment. Although Step Gage and lead 
screw have an identical coefficient of ex¬ 
pansion, to further guard against the effects 
of a temperature differential between them, 
the Step Gage is left to “soak” on the 
Measuring Machine table for 24 hours 
prior to calibrations. It is not removed for 



fig. 306 — The Step Gage is left on the 
Measuring Machine table until corrective 
lapping of the lead screw is complete. This 
practice is' also followed with jig borers 
and jig grinders for maximum efficiency 


and accuracy. Since many jig borers and 
jig grinders are being carried through 
calibration at one lime, this entails * large 

inventory of Step Gages. 
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9 SOURCES OF PERIODIC ERROR 


Thrust collar of lead screw flat and square to axis 



1 


Sleeve flat and square to axis 



2 




r^r 

Bearing 


A diameter 

<]T_ 





Tit end of screw 
Journal 


End journal support 
of lead screw must be on 
same axis and have good fit. 


7 


8 


9 


figs. 307 TO 315. No. 1 — The thrust collar 
must be jlat and square to the lead screw 
axis. No. 2—The thrust shoulder of the dial 
housing {containing support journal) must 
be flat and square to its axis. The balls 
which take the thrust must be round, and 
alike in size. No. 3 — Thread form must be 
perfect and uniform. No. 4 —Spacing of 
the lines of the dial must be uniform. 


Periodic error amounts to approximately 
1 /220th of the line spacing error. 

No. 5—Lead screw pilch diameter must, be 
uniform over the full length of the lead screw. 
No. 6—Misalignment of the lead screw 
causes both lead error and periodic error, 
but the, exact nature of the error is 
unpredictable. No. 7— The pilch diameter 
of the lead screw must be on the same axis 


as Ihe bearing diameter to avoid 
“crankshaft” error. No. 8 — Thread 
drunkenness, or departure from a true 
spiral, is commonly thought of as the only 
component of periodic error. In reality, it 
is only one of many sources. No. 9—The 
end journal support of the lead screw must 
be on the same axis as the lead screw and 
be a good fit. 















CORRESPONDING 9 TECHNIQUES OF ELIMINATING PERIODIC ERROR 



pigs. 316 T« 324 . No. 1 —The thrust collar 
of the lead screw is lapped flat and square 
to the axis of lead screw. No. 2 — The 
hearing sleeve thrust is inspected to assure 
flatness and squareness to the bore. 

No. 3—Form of the lead screw is viewed 
through the microscope and compared to the 
master 29° form. No. 4—Lines • [ the lead 


screw dial arc cut on the jig borer with a 
.003 inch \0.076 mm] wide saw, and the 
spacing is. controlled by a rotary table of 
±2 seconds accuracy. No. S — Pitch 
diameter of the lead screw is. inspected for 
uniformity. No. 6— The lead screw is 
inspected for concentricity of the pitch 
diameter to the bearing diameter. 


No. 7 — The slight thread drunkenness, 
unavoidable in thread-grinding, is removed 
by hand-lapping. No. 8 —A lapped test bar 
is substituted for the lead screw to inspect 
the alignment of the lead screw support 
journals. No. 9 — Inspection of the test bar 
at the far end from the dial housing support 
verifies the aliqnmeid of the end journal. 
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fig. 325— A wedge-shaped <jiuje, having 
precisely .10000 inch taper over 1 inch 
[The metric gage has 8 mm taper over 27 
mm], enables the lead screw to be inspected 
for periodic error anywhere over its full 
length. 
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the duration of the calibrations. This pre¬ 
caution necessitates a fairly large inven¬ 
tory of Step Gages, Fig. 306. 

The lead screw, after disassembly for cor¬ 
rective lapping, is put in a cleaning tank 
whose solvent is held to 68° F [20° C] as a 
means of rapid temperature normalization. 
Nevertheless, re-calibration does not take 
place until 24 hours after the lead screw is 
re-assembled. 

PERIODIC ERROR 

The term “periodic error” as applied to the 
lead screw refers to a constant repetitive 
error(s), occurring within one turn of the 
lead screw. Although usually a constant er¬ 
ror, repeating itself on each successive rev¬ 
olution, “periodic” can also gradually shift 
in phase over the length of a lead screw. 

“Periodic” is a disturbing type of error- 
first, because it can be of considerable mag¬ 
nitude; second, because it occurs within 
such a short travel; and third, because it is 
seldom given sufficient consideration by de¬ 
signers of machine tools and instruments. A 
common misapprehension is that periodic 
error is traceable only to thread “drunken¬ 
ness.” In reality, there are many potential 
sources of periodic error, as shown in Figs. 
307-315, inclusive. Only by adherence to 
strict standards of design and manufacture 
can its occurrence be prevented (see Figs. 
316-324, inclusive). 

Inspection for Periodic Error 
The Step Gage calibrates the error of every 
inch increment of the lead screw.* Although 
the lead checker has furnished a comprehen¬ 
sive calibration of overall lead error, in¬ 
cluding periodic, it is desirable to inspect 
the lead screw for effective periodic error after 
assembly, and to the same accuracy of de- 


*Metric equivalents are omitted by intention. 
The procedure for metric machines is similar, except 
that a wedge having 3 mm taper over 27 mm is 
used, and 6 points in one turn of the lead screw 
are checked. 


fig. 320 — The tapered gage used for 
checking periodic error is precisely aligned 
unth the machine axes. Note, however, 
that any slight misalignment is compensated 
for by charting the readings obtained. 


In this example, the gage has been 
misaligned by 10 millionths of an inch 
[i 0.25 pm]. The maximum error is at 
.300 inch where the reading was ± 8 
millionths of an inch [ 0.2 pm]. Actual 
periodic error at this point is approximately 
6 millionths of an inch [ 0.15 pan]. 
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FIG. 327 — Although the published guarantee 
is that jig borers and jig grinders be held 
within 90 millionths of an inch [0.0029 
mml on each axis, note that the lead screw 
lapping specialists are given a much 
narrower shop tolerance band. 


termination as accomplished with the Step 
Gage. 

A gage having precisely 0.10000 inch 
taper in one inch is mounted on the Mea¬ 
suring Machine table, Fig. 325. The pur¬ 
pose is to inspect the “Y” or cross axis 
for periodic error. Suppose that the “Y” 
axis displacement from 5.0000 inch to 
5.1000 inch is to be inspected. A reference 
surface of the gage (surface to the right in 
photo) is first aligned parallel to the “X” 
axis. The “Y” axis lead screw is set at 
5.0000 inches while the indicator is set to 
“zero” against one end of the tapered 
gaging surface (surface to the left in 
photo). Each nominal 0.100 movement of 
the “X” axis lead screw and 0.010 inch 
movement of the “Y” axis will again 
register the indicator against this surface. 
This process is repeated ten times over the 
length of this gaging surface. This inspec¬ 
tion yields a graph as shown in Fig. 326. 

Although in practice the gage is always 
precisely aligned, note that a slight mis¬ 
alignment does not effect the accuracy of 
the check. This method of inspecting 
periodic error necessitates good repeat¬ 
ability, and good machine “geometry.” 

MOORE LAPPING SPECIALISTS 

The guaranteed tolerance of the Moore 
Measuring Machine Master Lead Screw is 
35 millionths of an inch [0.0009 mm]. In 
fact, lapping specialists have a narrower 
working tolerance of + 12 j .-2 millionths of 
an inch [±0.0003 mm]. It is stipulated 
that only small, progressively minus lead 
errors be allowed, Figs. 327 and 328. 

There are three reasons for lapping the 
lead screw progressively “minus”: 

1. It is possible that from overzealous use 
by machine operators (applying more 
to jig borers and jig grinders, where 
the speed of movement is not moni¬ 
tored), that the lead screw will warm 
up slightly. However, a temperature 
rise would tend to make a “minus” 
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Note color area, which is the “tolerance area’’ 
for jig borers and jig grinders—total of 75 mil¬ 
lionths inch. Uneven line is actual check of Moore 
Jig Borer, Machine No. 356, sold to Alliance Tool 
and Die Corporation, Rochester, N.Y., on Septem¬ 
ber 28, 1960. 
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Uneven line is actual check of Moore Measuring 
Machine No. 122, sold to IBM, Military Products 
Division, Oswego, N.Y., on June 14, 1960. 






i 














i 































fig. 32S — The Measuring Machine lead 
screw is guaranteed to be within So 
millionths of an inch [0.0009 mm] overall. 
Note, however, that the nature of the 


narrower tolerance band given the lapping 
specialists f urther enhances the actual 
performance-accuracy of the Measuring 
Machine. 
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fig. 329 — In larger measuring machines, 
friction of movement is avoided by the use 
of roller-ways. The temperature of the lead 
screw is now more critical because of its 
greater length, so is controlled by being 
immersed in a balk of continually 
circulating oil. 



screw progressively longer. In other 
words, a slight temperature rise would, 
if anything, make the screw closer to 
zero. 

2. All Moore machines in the field will be 
alike. If machines were allowed to be 
indiscriminately plus or minus, they 
might disagree (in the case of jig 
borers and jig grinders) by 150 mil¬ 
lionths of an inch [4 /im], 

3. It permits only minute errors from 
inch to inch. The extremely good lin¬ 
ear characteristic of the Moore Lead 
Screwfitstherequirement that smaller 
measurements are necessarily of a 
higher order of accuracy. Note again 
Fig. 328. Any location between the 
fifth and twelfth inch [127 mm to 305 
mm] would have no more than 10 mil¬ 
lionths of an inch error [0.00025 mm]. 

Occasionally, closer accuracies are re¬ 
quired, and it is then necessary to entirely 
eliminate the slight build-up of heat in the 
lead screw. In this case, roller-way con¬ 
struction is employed to virtually eliminate 
friction of movement. 

In larger measuring machines, such as 
No. 4 and No. 5 designs, or for ruling en¬ 
gines, the temperature of the lead screw it¬ 
self must be maintained by immersing it in 
a heat sink of circulating oil, Fig. 329. 









DIVIDING 
THE CIRCLE 



The authority for “angles” of angle 
gages, rotary tables, or the angular re¬ 
lationships in mechanical parts and as¬ 
semblies is derived from the self-proving 
principle of dividing the circle. 


PIG. 330 —Mating elements of a disassembled 
1UW Index. Because the instrument is 
considered a principal angle standard and 
is of such exceptional accuracy, it is an 
appropriate symbol of the self-proving 
principle of circle-dividing to establish 
angles. 









1. The Nature of the 
Third Mechanical Art 

The third mechanical art to be considered 
is dividing the circle. A more appropriate 
designation of this art might be angles, 
since the measurement and description of 
angular magnitudes are fundamental to 
most aspects of engineering metrology. 

Angular relationships are inherent in 
geometric considerations (see pages 73-75). 
They are essential, for example, in the con¬ 
struction of masters andsquares,intheuseof 
angle-measuring autocollimators or levels, 
and in determining 90° relationships and the 
straightness of machine ways. Angles also 
come into consideration in measurements of 
length or size. Circle-dividing may be 
employed, for example, in the indexing of 
a lead screw, or in measuring the angle of 
a taper. 

Angle is frequently a prime consideration 
with mechanical parts—machine tool spin¬ 
dle tapers, gears, splines, cams and tapered 
bearings, to name a few. The measurement 
of angle may be accomplished in many ways 
with a variety of equipment from simple 
protractors and sine bars to precision 
instruments such as autocollimators, the¬ 
odolites, clinometers and electronic levels. 


Without surveying every technique by 
which angles are measured, this section will 
be concerned with the foundation on which 
the measurement of angles rests. Included 
will be descriptions of (1) the principles of 
establishing angle, (2) primary angle stan¬ 
dards and (3) accurate angle-measurement 
tools and instruments (see Fig. 330, which 
shows in disassembled form the mating 
segments of a 1440 Precision Index—a 
primary standard for dividing the circle). 

The analysis will show, in addition, that 
the division of the circle is so uniquely self¬ 
proving that it is distinct from other tech¬ 
niques of establishing angle. Moreover, this 
method of division is the only one inher¬ 
ently capable of deriving primary stan¬ 
dards. 

DIVIDING THE CIRCLE 
VERSUS THE SINE PRINCIPLE 

Any proof of angle can usually be traced to 
one of two methods used to generate angles 
—the sine principle and dividing the circle. 

The sine principle uses the ratio of the 
length of two sides of a right triangle in de¬ 
riving a given angle. Theoretically, any 
scale of units may be chosen for this pur¬ 
pose for it is the proportions, not the actual 
length of the sides, which determine the 
angle derived. 



fig . 'Six —The sexagesimal system is 
the accepted international method in science 
and engineering of relating angular 
magnitudes. This system is based on the 
numbers 3 and 6 and their multiples. 
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FIG. 332 —One historical explanation for 
the adaptation of the sexagesimal system 
■is that the ratios found in the numbers 
3 and 6 predominate, as is the case with 
triangles. Some ratios are shown. The 
number 3G0, moreover, has many factors. 


The second method is based on the fact 
that a circle can be divided into any num¬ 
ber of parts. The accuracy with which this 
is performed is proven once the circle is 
“closed.” 

Both methods, at least in theory, are 
capable of self-resolution. No outside au¬ 
thority such as a national bureau of stan¬ 
dards is necessary for final verification of 
angles obtained by either method. In prac¬ 
tice, however, the sine principle is depen¬ 
dent on an established system of length 
measurement. The degree of accuracy 
achieved is consequently dependent on the 
“traceability” of the length-measuring ele¬ 
ments which may be part of this system. 

To divide the circle into any number of 
parts, it is unnecessary to consult with the 
various national bureaus, though this may 
be done for credibility or for impartial veri¬ 
fication from a recognized authority. Un¬ 
like standards of length, where the magni¬ 
tude of units has been arbitrarily estab¬ 
lished, the circle can be divided into any 
number of units and be “self-proved.” The 
level of accuracy depends only on the pre¬ 
cision of available equipment and the tech¬ 
niques employed. 

DERIVATION OF PRESENT 
SYSTEM OF RELATING 
ANGULAR MAGNITUDES 

Today there is general acceptance of an 
established system wherein the circle is 
divided into 360 parts or degrees (°); each 
degree is divided into 60 parts or “min¬ 
utes” ('); and each minute is further di¬ 
vided into 60 parts or seconds ("), Fig. 331. 

The origin of the sexagesimal (units of 6) 
system pertaining to angles is usually 
credited to the Babylonians. It is some¬ 
times claimed that the number 360 was 
chosen because the Babylonians, like other 
ancient peoples, were sun worshippers. 
Supposedly, they calculated the days, or 
appearances of the sun within a year, as 



A triangle consistsof 6 parts; 
3 sides and 3 angles 



A 30° right triangle has 
angles of 30°, 60°, and 90° 
(a proportion of 1-2-3) 


360. Consequently, 360 became a sacred 
number. 

The writer believes that the calendars of 
the ancients were more exact than is recog¬ 
nized. A discrepancy of 5 days in calcula¬ 
tions of the number of days in a year is 
therefore unlikely—a viewpoint supported 
by some historians. Hawkins presents con¬ 
vincing evidence that early astronomers in 
England (circa 1300 B.C.) were able to 
compute important lunar and solar oc¬ 
currences (possibly eclipses) with amazing 
precision. * 

The sexagesimal system has gained such 
universal acceptance in the recording of 
time, in the measurement of angle, in geo- 
detics, in navigation and in all practical 
measurements for engineering, that an¬ 
other explanation of its acceptance is sug¬ 
gested. The number 3 and its multiples 
predominate in the mathematical and trig¬ 
onometric calculation of angles. A triangle 
is made up of six parts (three angles, three 
sides); a 30° right triangle has angles of 
30°-60°-90° (a proportion of 1-2-3); an 
equilateral triangle has three 60° angles, 
Fig. 332. 

*Gerald S. Hawkins, Stonehenge Decoded, New 
York, 1965. 



An equilateral triangle 
has three 60' angles 
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Toward the end of the 18th century, 
Lagrange introduced the “centesimal” 
(units of 100) system of angular unit. As in 
the sexagesimal system, the basic unit is 
the right angle . defined as the angle 
between two straight lines which intersect 
so as to make the adjacent angles equal.”* 
One would expect the right angle to be 
prominent in any system, since it is unique 
and self-proving (see page 34). 

The following compares the sexagesimal 
to the centesimal system: 

Sexagesimal System 
1 right angle =90 degrees (°) 

1 degree = 60 minutes (') 

1 minute = 60 seconds (") 
Centesimal System 
1 right angle = 100 grades (g) 

1 grade = 100 minutes (°) 

1 minute = 100 seconds (“) 

Some of the numerical conveniences of 
the sexagesimal system are apparent once 
there is an alternative system for compari¬ 
son. The factors in the number 360 are: 
1-2-3-4-6-8-9-10-12-15-18-20-24-30-36-40- 
45-60-90-120-180-360—twenty-three in all, 
compared to only twelve (1-2-4-5-8-10-20- 
25-50-100-200-400) in the four hundred 
grades which make a full circle in the cen¬ 
tesimal system. 

A right angle in the sexagesimal system 
is 90°, compared to one hundred grades in 
the centesimal system. When 90° is divided 
into thirds, the value is 30°, compared to 
33 l/3g (33g, 33°, 33 cc , 0.33...“), When 
it is further divided by three, the value is 
10°, compared to 11 l/9g. Many of the 
more important triangles, such as the 
equilateral triangle, are not described in 
terms of whole numbers in the centesimal 
system. 

Angular magnitudes may also be ex¬ 
pressed as a ratio of the length of the arc of 

*J. C. Evans & C. O. Taylerson, “Measurement 
of Angle in Engineering,” NPL Notes on Applied 
Science, p. 1. 


a circle to its radius. In this system, the 
unit is called the “radian”: “... the angle 
subtended by an are of length equal to the 
radius.”* In any given circle, therefore, the 
following relationship exists: 

Circumference of a circle 22 

- = — or 7r** 

diameter of circle 7 

then 2 n radians = 360 degrees 

1 radian = 57° 17' 44.8" 

— or — 

1 degree = 0.017453 radian 

The “millieme” or “mill” (1/1000 ra¬ 
dian) system is used by the United States 
Military, and is noted here to point out the 
existence of alternative systems of describ¬ 
ing angular magnitudes. 

General acceptance of the sexagesimal 
system as applied to angles may well be 
compared to Darwin’s theory of survival 
of the fittest. The survival of a species is 
related to its adaptability. Similarly, be¬ 
cause of its adaptability and greater math¬ 
ematical convenience, the sexagesimal sys¬ 
tem has survived and is the most commonly 
used. The centesimal system, on the other 
hand, is rarely used today. 

MATHEMATICAL 
INCONVENIENCE OF THE 
SEXAGESIMAL SYSTEM 

Under the sexagesimal system, if the circle 
is divided into 10 equal sectors, then each 
sector would have an angle of 360°/10, 
or 36°. In this case, the fractional relation¬ 
ship to the full circle is readily apparent 
and the angle easily calculated. However, 
to divide the circle into, say thirteen parts, 
not altogether an infrequent requirement, 
there are irritatingly inconvenient numer¬ 
ical values. The 13th part of a circle is 
(1/13) x (360°), or 27 9/13°, requiring con¬ 
version first to 27° 41 7/13' and then to 

*Ibid. p. 2. 

** j 2 is an approximation. 7r is actually equal to 
3.14159265. 



27° 41' 32". The 2/13th, 3/13th parts and 
so on are calculated similarly. 

Imagine the millions of additional hours, 
not to mention the toll from miscalcula¬ 
tions which have occurred over the cen¬ 
turies because of this clumsy system! There 
have been proposals to completely deci¬ 
malize the angle system, but as noted, 
more serious mathematical disadvantages 
are perhaps involved. A better compro¬ 
mise would be to decimalize only parts of 
a degree. For example: 352° 20' 24" = 
352.34°. In this manner, a great deal of 
convenience could be gained without dis¬ 
rupting a system which is fairly convenient. 

The foregoing analysis, seemingly ele¬ 
mentary, will be seen to be of fundamental 
significance in the design and use of circle- 
dividing equipment such as rotary tables. 

2. The Principle of 
Self-Proving the 
Division of a Circle 

The principle of self-proving the division 
of a circle is fundamental to metrology and 
is one with which the reader should be 
familiar before proceeding to an analysis 
of angle-measuring equipment. The prin¬ 
ciples involved are best understood by us¬ 
ing the familiar example of a pair of 
dividers to step off a circle. 

ESTABLISHING THE 
DIVISIONS OF A CIRCLE 
USING DIVIDERS 

Imagine for the moment that no angle 
standards exist, and that we have before 
us the task of establishing a master divided 
circle by using only simple tools. Assume 
that a circle A, Fig. 333, is to be divided 
into ten equal sectors. 

The dividers are first set to an estimated 
tenth part of the circle. If the dividers are 
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carefully stepped off around the circle and 
closed on the starting point, the tenth step 
will be long or short (plus or minus) ten 
times the original setting error. By simply 
adjusting the dividers slightly to eliminate 
only part of the error, and repeating the 
stepping-off process again and again, the 
circle comes ever-closer to being divided 
into exactly ten sectors. Finally, a scribed 
line or some other mark can be made to 
denote the ten divisions. 

A slight variation in procedure divides 
the circle into ten equal parts much more 
rapidly. Only one trial stepping-off of the 
circle is made. When the circle is closed on 
the tenth step, the accumulated error is 
measured exactly, using a graduated rule, 
for example. The dividers are finally ad¬ 
justed by precisely l/10th the total error, 
and the circle marked in ten equal sectors 
after only one trial. 

THE THEORY OF 
CALIBRATING A 
DIVIDED CIRCLE 

Let us now suppose that circle A has been 
divided into ten sectors, nominally 36°, as 
accurately as is considered practical by 
using dividers, and that it is known there 
have been errors in placing the marks de¬ 
noting each sector. The usefulness of that 
divided circle may be further enhanced if 
the error of each of the ten sectors is exactly 
determined or calibrated. 

In a first method of calibration, the 
dividers are set to a nominal 1/10 part of 
the circle, either in reference to one of the 
sectors of the circle considered most cor¬ 
rect, or to a 1/10 sector of some other mas¬ 
ter divided circle. It is important to note 
that this sector need not represent exactly 
36°. 







Let it be assumed that the dividers are 
set equal to sector A, and the circle is 
once again stepped off. The accumulated 
difference between A and each of the other 
sectors is recorded, as well as the total 
accumulated error on closing. Once the 
initial setting error at A is determined 
(closing error divided by 10) it is applied to 
determine the true calibrated value of each 
sector, Fig. 334. 

A variation of this method is more 
usually applied to the calibration of circle- 
dividing equipment. Again, A may be des¬ 
ignated temporarily as master X, whose 
true value is as yet undetermined. Sector 
X is not stepped off but only compared to 
each of the sectors A,B,C,D,E,F,G,H,I,J. 
Each difference is measured exactly and 
recorded. The value of X is now deter¬ 
mined algebraically and applied to derive 
the true angle of all the included sectors, 
Fig. 335. 

The previous techniques may be applied 
to establish or to calibrate any chosen divi¬ 
sion of the circle, whether it be 13, 20, 160, 
or 1440 parts. The verification of the most 
accurate circle-dividing instruments relies 
ultimately on such elementary proofs, 
regardless of the expressed magnitude of 
error. 


3. The Sine Principle 


First the value of angle “A” is calculated: 


10A=380” (360° + 20°) 

A = 38” (or 2” error) and applied to column (5) 


(1) 

Angle 

(2) 

Nominal 

included 

angle 

(3) 

Measured 
difference 
(at each step) 

(4) 

Apparent 

included 

angle 

(5) 

Applied 
correction 
for angle A 

(6) 

Actual 
included 
angle from 
original setting 

(7) 

Value of 
sector 

Error of 
sector 

A 

36° 

0 

36° 

+ 2° 

38° 

38° 

+2” 

B 

72” 

- 4° 

68° 

+ 4° 

72° 

34° 

-2" 

C 

108° 

- 5' 

103° 

+ 6° 

109° 

37” 

+ 1° 

C 

144° 

- 8” 

136° 

+ 8° 

144° 

35° 

-1° 

E 

180° 

- T 

173° 

+ 10° 

183° 

39° 

+ 3” 

F 

216° 

-12* 

204° 

+ 12° 

216° 

33° 

-3” 

G 

252’ 

-14° 

238° 

+ 14° 

252° 

36° 

0 

H 

288° 

-20° 

268° 

+ 16° 

284° 

32” 

-4” 

1 

324° 

-19° 

305° 

+ 18° 

323° 

39° 

+ 3° 

J 

360° 

-20° 

340° 

+ 20° 

360° 

37” 

+ 1” 


It has been observed that, in addition to 
the principle of dividing the circle, angles 
may be established or calibrated through 
use of the sine principle. The practical 
application of the sine principle for the 
measurement of angle will be analyzed 
here with respect to four essential points: 

1. Devices that operate on the sine 
principle, just as those that divide 
the circle, are capable of “self-gen¬ 
eration.” 

2. Unlike the division of the circle, 
angles established by means of the 


FIG. 334 — Using dividers, the value of each 
of the U) angular sectors of the above circle 
is calculated or calibrated by comparison to 
a master sector which itself is stepped off 


around the circle. The value o} the master 
sector is determined on closing by 
measuring Us accumulated error. 
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Recorded errors of sectors ( 1) A—x= 0° 

( 2) B-x=-4” 
( 3) C—x= — 1” 
( 4) D—x= —3° 
( 5) E-x= + l" 
( 6) F-x=-5° 
( 7) G-x=-2" 
( 8) H—x= —6° 
( 9) l-x= + l" 
(10) J-x=-l° 

Adding (A + B + C + D + E + F + G + H + l + J)-10x = -20" 
But(A + B + C + D + E + F+G + H + l+J) = 360" 
Substituting 360" —10x = —20° 


Substituting x=38°: ( 1) A — 38"— 0" A=38" 

( 2) B—38°— —4" B = 34" 

( 3) C-38" = -l" C=37° 

( 4) D—38"= —3" D=35° 

( 5) E —38"= +1" E = 39° 

( 6) F—38"— —5" F = 33" 

( 7) G-38° = -2° G = 36" 

( 8) H —38"— —6" H = 32" 

( 9) l-38"= + l" 1 = 39" 

(10) J —38"= —1" J = 37" 

fig. 335— Calibration of the 10 sectors difference at each step. The value of the 

of the above circle may proceed by master sector is determined algebraically 

comparing a master sector “X” to each of and then applied against the readings 
the other sectors, and measuring the obtained to find true error of each sector. 
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FIG. 336— Some elementary concepts 
relating to angles: left, the formation of an 
angle; center, the formation of a triangle; 
right, the numerical valve in degrees of 
the included angles of a triangle. 


fig. 337 —(center left ) Shape of a triangle 
is not known until the magnitudes of at 
least two of its included angles are specified. 

FIG. 338 — (center right) Size of a given 
triangle is not known until the length of 
one side is specified. 




A + B + C= 180" 
70° + 80°+C= 180‘ 
then 

C= 30° 




sine principle are limited in practice 
to only 90° (1/4 of a circle). 

3. Devices that operate on the sine 
principle, even within their 90° useful 
range, in practice diminish in accu¬ 
racy the greater the angle. The loss 
of accuracy is particularly rapid after 
the angle exceeds 45°. 

4. The accuracy with which the sine 
principle can be put to use is de¬ 
pendent, in practice, on some form 
of linear measurement. 

It is not our purpose to delve at length 
into geometric or trigonometric relation¬ 
ships. However, the practical application 
of the sine principle in engineering metrol¬ 
ogy does require an understanding of a 
few simple premises: the intersection of 
two straight lines forms an angle. A tri¬ 
angle is formed when the two sides of this 
angle are joined by a third straight line, 
Fig. 336. Included angles A+B+C always 
total exactly 180°. The shape of the result¬ 
ing triangle is not known until the magni¬ 
tudes of two of its angles are specified, Fig. 

337. The relative size of the triangle is not 
known until, in addition to two angles, the 
length of at least one side is specified, Fig. 

338. It can also be seen that the length of 
two sides does not help us to determine 
angle, Fig. 339. When the length of the 
third side is specified, then the three in¬ 
cluded angles may be determined. 


THE RIGHT TRIANGLE 

It has been shown (see “Geometry Sec¬ 
tion” ) that the 90° angle is the basis of the 
orthogonal relationships which exist in the 
movable members of a machine tool; also, 
that the right angle is uniquely self-proving 
by reversal methods. 

The right triangle is of particular in¬ 
terest to engineering metrology since it 
allows a simple, convenient calculation of 
most of the angular and dimensional rela- 


fig. 339 —Specifying the length of two 
sides of a triangle is not sufficient to 
determine angle. 'The length of the third 
side must also be known. 


FIG. M0— The right triangle is particularly 
useful in determining angular magnitudes; 
the principle is applied in such tools as 
sine bars, sine blocks and sine plates 
to establish angle. 
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tionships commonly encountered. These 
relationships can be expressed in terms of 
trigonometric functions, usually given as 
follows, Fig. 340. 

Where Sine C = or 

„. . , , opposite side 

Sine of the angle = --- 

hypotenuse 

Cos C = or 
b 

„ . , ,, , adjacent side 

Cosine of the angle = --- 

hypotenuse 


Tan C = - or 
a 


Tangent of the angle = 


opposite side 
adjacent side 


The inverse ratios are given as: 

a 


Cot C = - or 
c 

Cotangent of the angle ; 


adjacent side 
opposite side 


Sec C = - or 
a 


Secant of the angle 


hypotenuse 
adjacent side 


Csc C = - or 
c 


Cosecant of the angle ; 


hypotenuse 
opposite side 


These relationships are useful in the cal¬ 
culations of angles or dimensions. If, for 
example, angle C and side b are given, then 
the length of side a is derived by the 
formula: 

Cos C = °r 
b 

—or— 

Cos C X b = a 

If the lengths of sides b and c are known, 
then the angle at C may be determined by 
calculating the sine, using the formula, and 
then referring to a table of trigonometric 
functions. 


THE PRACTICAL APPLICATION 
OF THE SINE PRINCIPLE 

Three important angle-measuring devices 
are based on the latter sine relationship— 
the simple sine bar, the sine block and the 
sine plate. 

The sine bar and sine block are essen¬ 
tially single elements, and are mounted 
on an auxiliary flat surface. The sine bar is 
primarily a gage or angle reference. The sine 
block is designed to make angular measure¬ 
ments of mechanical parts mounted upon 
it. The sine table as with the sine block, 
is meant to measure angles of workpieces. 
Unlike the other two devices, however, 
the sine table consists of two elements— 
a base and a table. These are permanently 
hinged at the sine angle. All the legs of 
the right triangle are self-contained within 
the sine table. Since the three devices are 
closely related, the principles involved can 
be demonstrated using the sine bar as an 
example. 

The Sine Bar 

The sine bar is composed essentially of one 
piece or bar, Fig. 341. The hypotenuse, or 
side b is established in construction as the 
center-distance of two hardened gage pins. 
The desired angle C is set according to the 
formula: 

. „ c 

sine C = -r 
b 

If the angle desired is 30°, and a 5-inch 
sine bar is used the dimension at c is: 

c = Sine C X b 
c = 0.500 X 5 inches 
—or— 

c = 2.500 inches 

Most commonly, a stack of gage blocks 
is set in the proper combination to form 
side c, or the right angle. Any convenient 
plane surface may be selected to support 
sine bar and gage blocks. Preferably, the 
surface should be hardened, ground and 



fig . 341 — The simple sine, bar is an 
example of the use of the sine relationship 
to establish angle. The right triangle is 
shown superimposed. 
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fig. 342 —Angular setting devices which 
operate on the sine principle, inherently 
become increasingly more inaccurate 
as the angle exceeds 4-5°. Depending on 
design, many become virtually impractical 
to use, as with the sine bur set at 80°. 



lapped to a high degree of flatness. The 
use of gage blocks is not absolutely neces¬ 
sary, since various other convenient length¬ 
measuring tools may be employed. 

It is the ratio of the lengths which is 
important, not the actual lengths of the 
units involved. If metric units are used, the 
same ratios are still obtained. Provided that 
all units are accurately subdivided and ap¬ 
plied to both legs of the triangle, any system 
of length-measurement is applicable. It is 
most convenient to utilize the established 
inch or metric systems for sine bar measure¬ 
ments, since accurate gage blocks and other 
types of length-measuring devices are read¬ 
ily available. 

The sine bar is not a complete measuring 
instrument in itself. It only presents its 
top surface, or that of some mounted work- 
piece, to the required angle. Another 
datum such as a surface plate or machine 
axis is needed, as well as other auxiliary 
equipment, notably indicating devices, to 
make measurements. 

Historically, the use of the sine bar to 
establish angle was considered preferable 
to the use of circle-dividing equipment. 
Preference for the sine bar can possibly be 
explained by three factors—its simplicity, 
the great progress made in length-measur¬ 
ing techniques since the introduction of 
gage blocks by Johansson, and the fact 
that it is only in recent years that circle- 
dividing instruments and allied equipment 
have been made to any degree of perfection. 


LIMITATIONS OF THE 
SINE PRINCIPLE 

The establishment of angle by the sine 
principle is essentially a length-measuring 
process. Therein lies its greatest virtue and 
its greatest weakness. The virtue of the 
sine principle for the measurement of angle 
is that very accurate end standards (gage 
blocks) are readily available, and can be 


put together in almost any desired in¬ 
crement to establish the length of the side 
containing the right angle. 

The factors which limit the accuracy 
of the sine principle, in practice, are typ¬ 
ical of those encountered in almost any 
length-measuring situation. The length of 
the sine bar is represented by the center 
distance (normally 5-inch, or 10-inch) of 
two precision rollers. Although it is not a 
required condition that these rollers be to 
some exact nominal size, they must be 
alike in size, and round, parallel-sided, 
parallel to each other, and parallel to the 
working surface of the bar. The required 
accuracy of these functional elements is 
defined in Commercial Standard No. 
141-47 issued by the United States Depart¬ 
ment of Commerce in cooperation with the 
National Bureau of Standards. 

In “Standards of Length” (pages 129- 
131), was discussed the difficulty of measur¬ 
ing the true size or length of a geometrical 
shape other than the separation of two 
parallel planes. To measure the size of a 
simple single gage pin becomes a complex 
problem. It should now be apparent that 
the geometrical condition involved in mea¬ 
suring the exact, effective center distance ex¬ 
isting between two rollers of the sine to a cer¬ 
tainty of a few millionths of an inch [fraction 
of a jum) is an infinitely complex problem. 
This fundamental limitation alone pre¬ 
cludes the use of the sine bar as a primary 
standard of angle. 

Devices which operate on the sine prin¬ 
ciple are fairly reliable at lower angles 
(i.e., less than 15°) but become increasingly 
inaccurate as the angle increases. This 
characteristic is best understood by analyz¬ 
ing the operation of the sine bar, as well 
as some of the trigonometric relations which 
exist at different angular settings. 
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At an angle of 10°, the required stack of 
gage blocks is small, hence measurements 
are fairly reliable. Whatever in-built error 
may exist in the length of the sine bar is 
not critical. For example, at this angle, an 
error of 0.0001 inch [0.0025 mm] in the 
center-distance of the rollers of a 5-inch 
sine bar [127 mm] would introduce an error 
of only 0.728 second. It should be noted 
also that the gage block increments are 
almost linear. The difference in height of 
the gage block stack for a 5-inch sine bar 
from 10° to 10° 30' is 0.0429365 inch 
[1.0906 mm], and from 10° 30' to 11° is 
0.0428675 inch [1.0888 mm]. 

Compare now the conditions when the 
sine bar is set at 80°.* As evident in Fig. 
342, sine bars inherently become increas¬ 
ingly impractical and inaccurate as the 
angle exceeds 45°. The reasons are: 

1. The sine bar is physically clumsy to 
hold in position. 

2. The body of the sine bar obstructs the 
gage block stack, even if relieved as 
shown in Fig. 342. 

3. Slight errors of the sine bar cause large 
angular errors. For example, at 80°, 
an error of only 0.0001 inch [0.0025 
mm] in the separation of the rollers 
causes an error of approximately 23.3 
seconds, or 32 times greater than the 
error when at 10°. 

4. Long gage blocks or stacksare not nearly 
as accurate as gage blocks shorter than 
1 inch. 

5. Temperature is more critical. It is now 
absolutely essential that the tem¬ 
perature of the sine bar body sepa¬ 
rating the two rollers be identical to 
that of the selected stack of gage 
blocks. A disparity in temperatures 


*Sine bars should not be set to angles over 45°. 
Instead, they should be used in conjunction with a 
mounted square. This analysis does apply to the 
sine table, which often is set over 45°. 


may exist from handling the gage 
blocks, or possibly from storing the 
gage blocks in an environment warm¬ 
er or colder than where the sine bar 
is to be used. 

6. A difference in deformation occurs at 
the point of roller contact to the sup¬ 
port surface and to the gage blocks. 
Assuming that the relative load on 
each roller is equal and constant, no 
appreciable difference in deformation 
occurs. However, as the angle in¬ 
creases, the weight-load is shifted 
more toward the fulcrum roller (an¬ 
gle C). The difference in deformation 
becomes quite appreciable if work- 
piece weight is added. Such is the 
case with the sine block. 

The Sine Table 

The size of gages, instruments or parts that 
a sine bar can inspect is limited, since it is 
not designed to support large or heavy ob¬ 
jects. The sine block, being of one-piece 
construction, is also limited since it may 
safely carry workpieces only to about 45°. 

The sine table is the most convenient 
and accurate design for sustaining rela¬ 
tively heavy loads, since (1) the weight of 
the unit plus that of the workpiece is given 
fuller, safer support; (2) the gaging plat¬ 
forms are self-contained and can be highly 
refined; (3) the table may be safely swung 
to any angle from 0° to 90° by pivoting it 
about its hinged end. 

It is exceedingly difficult to absolutely 
verify the angle set by a sine table. Some of 


the reasons for this have already been 
shown to be inherent in the nature of the 
sine principle itself. The sine table has the 
additional problem of size and weight of 
its own table. In effect, the table is a long 
lever which bends and twists when put 
through various angles while supporting 
all sorts of shapes, sizes and weights of 
workpieces. The clamping mechanism of 
the sine table may also cause distortion, 
varying the angle from that intended. 

Unfortunately, these errors go undis¬ 
covered unless they are relatively large. 
Only two angles, 0° (flat) and 90° (right 
angle)—the limits of travel—are verified 
with any amount of certainty. At all other 
angles within those limits, even angles 
accessible to inspection, any known means 
of inspecting the sine table are not ap¬ 
preciably more reliable than the sine table 
itself. 

The best recourse for the sine table user 
is simply to satisfy himself that: (1) the 
construction principles of the sine table are 
correct; (2) all elements of the sine table 
are made accurately; (3) the sine table is 
used properly; and (4) the gage block stack 
is accurate and at the same temperature as 
the sine table. Having determined these 
things, all the user can do is trust the de¬ 
vice. 

The 8-inch [203.2 mm] sine table shown 
in Fig. 343 was designed to avoid the po¬ 
tential sources of inaccuracy normally as¬ 
sociated with a sine table. It is of interest 
that the use of integral gage pins in base 
and table, rather than a gaging platform, 
enables up to a full 90° without obstruc¬ 
tion. The “double sine” design, Fig. 344, 
also minimizes the errors normally inherent 
in a sine instrument involving greater 
angles. At 80°, for example, an error of 
+0.0001 inch [0.0025 mm] in the center 
distance of the gage pins causes an error of 
only —4.33 seconds. The double sine de¬ 
sign is robust. Its base and table are each 
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FIG. 343*-- Thin sturdy Micro-Sine Table 
departs from standard design by being 
gaged and supported on both sides. In this 
manner, accurate angles and parallelism 
of its table to base at all heights is assured. 



Setting for 
adjustable 
gage block 


.6161 diameter gage pin (2) 


All dimensions are in inches 


1 15/16 inches [49.21 mm] thick and it 
weighs 130 lbs. [58.97 kg]. Total profile 
height and weight are kept at a minimum 
(commensurate with rigidity) to achieve 
the greatest possible convenience and 
versatility in use. 

The sine table may be used for the in¬ 
spection of angular relationships while 
mounted on a surface plate, Fig. 345. It 
may also be mounted on a machine tool or 
measuring machine by itself or with a 
rotary table for the inspection of com¬ 
pound angles (see page 315). The low pro¬ 
file of the sine table avoids using up 
spindle-to-table capacity when it is mounted 
on jig borers, jig grinders or measuring 
machines. Two sets of sturdy noninfluenc¬ 
ing clamps are provided for supporting the 
table on both sides over the whole range of 
the sine table. 

The most important innovation in its de¬ 
sign is the incorporation of a pair of pro¬ 
truding gage pins set at the 8-inch [203.2 
mm] sine distance on each side of the plate. 
A special, four-spindle machine accurately 
locates the holes which are to receive the 
protruding gage pins, Fig. 346. A series of 
rods, threaded at both ends, enables very 
fine height adjustments to be made over 
the full 90°. Rapid settings may be made 
where great accuracy is not required, sim¬ 
ply by working to a graduated reference 
plate on the side of the sine table, or by 
gaging the height with vernier calipers. 

The sine table is capable of exceptional 
accuracy. The user should first be satisfied 
with the accuracy of the particular gage 
block stack in use, handling them with 
gloves. The sine table is elevated or low¬ 
ered, using the fine adjustment feature to 
attain the desired “feel” between the gage 
blocks and pins on one side. This side is 
then clamped, a step repeated on the other 
side. At this time, the operator may com¬ 
pare both sides to insure they are identical 
in height and to make any final minor ad¬ 
justment. 


fig, 341 —The “double-sine” principle 
employs gage pins rather than a gaging 
platform in both table and base of the sine 
table. This design allows angular settings 


to a full 90°, and minimizes the errors 
normally inherent in a sine table at 
greater angles. 
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FIG. 340 —Special 4 .-spindle machine used 
to establish location of 4 functional holes 
of both, table and base of the Mic.ro-Sine 
Table. 


FIG. 345 - Micro-Sine Table is usef ul for 
surface plate work in measuring angular 
relationships. Here, a 45° angle attachment 
used on jig grinders for generating ball 
sockets is inspected. 
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FIG. 347 —The rotary table, the beat-known 
and most widely mpplied circle-dividing 
instrument. 




4. Circle-Dividing 
Instruments 

THE ROTARY TABLE 

There are innumerable types of instru¬ 
ments, optical devices and mechanical 
contrivances which generate or measure 
angles by dividing the circle. There is none, 
however, which has more general use and 
significance to practical engineering and 
metrology than the familiar rotary table, 
Fig. 347. 

The three categories of rotary tables are: 

1. Optical, or other types having circu¬ 
lar scaled graduations, Fig. 348. 

2. Mechanical, cam-compensated, Fig. 
349. 

3. Mechanical, accurate worm-and- 
gear, Fig. 350. 

Optical Rotary Tables 

The term “optical” is a slight misnomer, 
and is used here only because this primary 


type of rotary table is usually thus de¬ 
scribed. It is optical only in the sense that 
a microscope is employed to read the essen¬ 
tial measuring element, a precision circular 
scale. Within this category may also be in¬ 
cluded other circular scale-measuring ro¬ 
tary tables which employ electrical (in¬ 
ductance) or optical (Moire fringe) averag¬ 
ing of the circular graduations. All differ 
slightly, but have similar characteristics. 

The number of lines which can be placed 
about the main circular scale of an optical 
rotary table depends on its diameter, the 
width of the lines and the magnification of 
the integral microscope. The lines are 
usually at intervals of % or of a de¬ 
gree. A graduated drum or other optical 
vernier arrangement enables settings to be 
made to finer increments of minutes and 
seconds of arc. In the “averaging” type, 
smaller increments of angle are achieved 
by the sine curve generated by electronic 
or optical pulsing. 

Since the circular scale or disc is usually 
manufactured separately before being 
mounted on the rotating member of the 
rotary table, the most advanced systems 
read the circular scale simultaneously from 
opposite sides to compensate for errors 
which may occur from eccentric mounting. 
Usually, a worm-and-gear is used to im¬ 
part rotation, but is not made to any great 
accuracy. 

Two advantages attributed to the opti¬ 
cal rotary table are: (1) the measuring ele¬ 
ment is free from wear and (2) the table 
may be positioned from either direction. 
Some of the disadvantages are: (1) its re¬ 
duced accuracy or inability to be employed 
for automatic indexing when using an in¬ 
dex plate (see pages 218-220); (2) eye 
fatigue from prolonged use (if optical).* 
“Repeatability” is also dependent on the ro¬ 
tary table’s ability to make very fine dis- 


*Much of this analysis is similar to that where 
various linear measuring elements are compared in 
“Standards of Length,” pages 185-186. 
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FIG. MS— The optical rotary table employs 
a circular scale read by microscope as its 
angle-measuring element. 


placements and this, in turn, is dependent 
on many factors, such as the type of radial 
and axial bearing used, lubrication, and the 
perfection of the total worm-and-gear assem¬ 
bly. 

Mechanical, Cam-Compensated 
Rotary Table 

In this type of rotary table, accuracy is 
directly related to the accuracy of the 
worm-and-gear, except that the final unre¬ 
solved errors are overcome by compensa¬ 
tion, rather than by perfection of the gear. 

Attached to the rotating member is a 
circular cam. Its shape is adjusted accord¬ 
ing to the calibrated error of the rotary 
table. Working against the cam is a fol¬ 
lower which, through a linkage, advances 
or retards the vernier dial or “zero” to 
match the error of the gear at that particu¬ 
lar angle. For example, if at 230°, the in¬ 
herent gear error is —10 seconds, then the 
cam advances the “zero” 10 seconds at that 
point, causing the operator to feed the worm 
through an additional 10 seconds in setting 
to his dial reading. With proper design, this 
system can have fairly good repeatability. 

The greatest advantage of the cam- 
compensated rotary table is the fact that it 
can offer the user reasonable accuracies at 
minimum cost. This principle of construc¬ 
tion has a limited potential of accuracy, 
however. The primary reason is that 
periodic error (error within one turn of the 
handwheel) is not taken into considera¬ 
tion. This is the most difficult error of all to 
eliminate, since it is effected by so many 
factors—the thread form, drunkenness, 
pitch, lineup, and runout of the worm, and 
also by the errors of eccentricity, tooth form, 
and tooth-spacing of the gear.* The errors 
of the worm alone, exclusive of the gear 

*The reader might refer to the analysis of periodic 
error of a lead screw, pages 196-199, since the com¬ 
parison is similar. The main difference between the 
two conditions is that periodic error is much more 
difficult to control in a rotary table than with a lead 
screw due to the lack of any averaging. 





FIG. 349 (Center) — In the cam-compensated 
rotary table, the angle-measuring element 
is the worm-and-gear. Final accuracy, 
however, is established by cam-compensation 
of gear error. 


fig. 350 (Bottom) —In the accurate 
worm-and-gear rotary table, the gear is 
machined »r lapped to final accuracy. 
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error, may easily amount to many seconds. 

The very process of resolving periodic 
error involves an expensive, exhaustive in¬ 
spection of at least several hundred posi¬ 
tions. One would not expect this exhaustive 
checking of a design which has been made 
with minimum cost the prime considera¬ 
tion. It is also doubtful that periodic error 
could be corrected by a cam and follower. 
For example, consider a 180-tooth, 10-inch 
[254 mm] diameter rotary table. One turn 
of the worm represents an arc in the cam of 
only about 3/16 inch [4.75 mm]—a very 
small space within which the follower must 
work. 



Cam-compensation may be a suitable 
means of removing gross errors. When ac¬ 
curacies to 1 or 2 seconds are required, 
however, the action of the cam, follower 
and linkage itself is likely to introduce se¬ 
rious errors. The cam and linkages are also 
vulnerable to chips and dirt if exposed, 
and are subject to wear if unhardened. 

Accurate Worm-and-Gear 
Rotary Table 

Both the worm and the gear of this type of 
rotary table are made highly accurate and 
no corrective mechanism is applied. One 
disadvantage of this design is the difficulty 
and expense of manufacturing an accurate 
worm-and-gear. The unavoidable presence 
of backlash, necessitating a unidirectional 
approach when setting, is also frequently 
cited as a disadvantage of this design as 
well as of the cam-compensation design. 
The writer takes issue with such criticism, 
based on personal experience in the ma¬ 
chine shop. A habit worthy of cultivating is 
that of always taking a unidirectional ap¬ 
proach where practicable, whether the 
device operated is mechanical, optical or 
electronic. 

The reader may satisfy himself on this 
point if he has occasion to test a measuring 
instrument that is supposedly free of re¬ 
versal error. Simply employ some auxiliary 
means (such as an electronic indicator and 
a suitable gaging surface) and compare the 
range of readings obtained with settings 
from one direction, then from the other. 
The discrepancies are sometimes great, 
sometimes slight, but almost always 
measurable. 

The advantages of the accurate worm- 
and-gear are: (1) trouble-free operation; 
(2) good repeatability; and (3) a high po¬ 
tential of accuracy. 

Advantage of the Index Plate 

The accurate worm-and-gear has one addi¬ 
tional advantage for the generation and 


FIG. 351 — The accurate worm-and-gear 
type rotary table is unique in that it may 
be used together with an index plate to 
divide the circle, without the necessity of 


reading angles, and without loss of 
accuracy. Here, 500 lines of a dial are 
being accurately generated. 
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inspection of angles. It was previously 
stated (see page 206) that the potential 
mathematical inconvenience of the official 
sexagesimal system might influence the de¬ 
sign and type of circle-dividing equipment 
selected as well as its manner of operation. 
A practical example will illustrate what 
was meant. In Fig. 351, five hundred lines 
0.003 in. thick [0.0762 mm] are cut, 
equally spaced around a 634 inch [165.1 
mm] diameter workpiece. The calculations 
necessary to generate or inspect these lines 
are: 

1 18 ° 

First setting — X 360° = -f- = 43' 12" 
500 25 

n ofjo 

Second setting—— X 360° = — = 1°26'24" 

500 25 

Q K4° 

Third setting ~ X 360° = Af = 2° 9' 36" 
500 25 

Each position must be read off the dial. In 
calculating the angles, or in setting these 
angles on the dials, an error can easily oc¬ 
cur and go undetected. 

The laboriousness of having to read the 
angles is in sharp contrast with the conven¬ 
ience of the index plate method. The value 
of the five hundred settings do not really 
have to be calculated. The rotary table 
used for generating the lines has a 180- 
tooth gear (2° per turn), so the formula is: 
180/500=9/25. 

This means simply that a 25-hole index 
plate is used and that the operator indexes 
ahead nine holes at a time. In use, the 
handwheel is replaced by an index plate 
having a 25-hole pattern and an index 
attachment. 

In summary, the advantages of using 
the index plate are: 

1. All that is required is a repetitive 
movement of the indexing pin, nine 
holes ahead per setting, aided by sec¬ 
tor arms locked at that spread, Fig. 
352. 

2. No readings are necessary and no 
special skills are required. 




Step 1. 

Index plunger moves 
clockwise to sector arm 



Step 2. 

Sector arm moves 
clockwise to plunger pin 


pig. 352 —For either the inspection or the 
generation of angles, the index plate aids 
accuracy and reliability. Here, the sector 
arms are set to enable 500 divisions of a 
circle using 9 holes at a time of a 25-hole 
plate. The divisions are self-proved upon 
closing of the circle. 






3. If the circle closes during a trial run, 
the setup is proved. 

4. An indexing error which may have 
occurred in any one of the five 
hundred steps during the operation 
does not go undetected, because the 
circle does not close. 

5. At any indexed position, angular 
location can be double-checked 
against the dial reading. 

6. The accuracy of the index plate is 
not critical, but need only be of 
second-order accuracy. An error of 
1.6 minutes in the index plate results 
in only about 0.5 second actual error 
in table position. 



Normally, a cast-iron index plate which 
has been drilled and reamed is furnished. 
In its place may be used a hardened, jig- 
ground index plate in which all holes are 
held to close tolerance of size, spacing and 
concentricity with its locating hole on the 
worm-shaft. In the latter case, repeatabil¬ 
ity far exceeds the ability of the eye to set 
by using the dial. 

The optical rotary table is not appro¬ 
priate for index plate use, since no effort 
has usually been put into refining its worm- 
and-gear. Similarly, the usual method 
employed with a cam-compensated rotary 
table is to shift the zero dial. In using an 
index plate, then, accuracy reverts back to 
that of the worm-and-gear, making the 
compensation mechanism completely in¬ 
effectual.* 

An indexing attachment can be used 
with the accurate worm-and-gear without 
sacrificing the optimum accuracy of the 
rotary table. This is not true of the other 
designs. Use of indexing attachments al¬ 
lows the accurate worm-and-gear design 
to be adapted with little or no loss of ac¬ 
curacy to many automatic or automated 
operations simply by controlling the whole 
and fractional number of worm rotations. 
For example, the worm may be indexed 
accurately by attaching a crank arm to the 
end of the shaft, using it to stop rotation 
positively by registering against an anvil. 
Another method is to use a stepping 
motor to control indexed position. In the 
simplest arrangement, the stepping motor 
is fastened directly to the wormshaft. For 
finer discrimination of angle, intermediate 
reduction gearing is used between worm- 
shaft and stepping motor. 

The angular increments set by the step¬ 
ping motor may be uniform, which is most 

*The writer has seen cam-compensated rotary 
tables that shift the whole dial housing, which would 
automatically correct for index plate use as well. 
This design would seem to have a low potential of 
accuracy, since worm alignment would certainly be 
altered during the rotation of the housing. 


FIG. 353 — The accurate worm-and-gear type 
rotary table allows simple indexing controls 
to be applied, as with this stepping motor. 


The rotary table is indexed antdrriatically 
t$ measure non-uniform line spacing of a 
dial, using a 30-power microscope,- 
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fig. 35-1 —The accurate -performance of a 
rotary table is made possible only by 
controlling the accuracy of all the 
mechanical parts which make up its 
assembly. 


often the case when machining or inspect¬ 
ing a circular cam. By applying additional 
controls to the stepping motor, Fig. 353, a 
rotary table has been programmed to en¬ 
able the inspection of nonuniformly spaced 
lines scribed around a circular part. The 
microscope used is 30-power. 

A ruling engine for the generation of 
diffraction gratings is shown in Fig. 555. 
The final performance of the engine de¬ 
pends to a large extent on an accurate 
master worm-and-gear, which is applied 
to index the leadscrew automatically, 
allowing table displacements as fine as 
0.0000001 in. or 0.1 millionth of an inch 
[0.0000025 mm]. 

Construction of the Rotary Table 
Only by closely controlling all mechanical 
elements which make up a rotary table can 
a high order of angular and rotational 
accuracy be attained. Fig. 354 shows the 
construction features of the disassembled 
parts of a rotary table. The manufacturing 
tolerances of these parts as listed below 
should be noted (read from left, clockwise). 



1. Hardened, ground and lapped (ID) 

bearing sleeve. 

2. Cast iron base (47 lbs. [21.3 kg]) 

scraped flat. 

and parallel. 

3. 9% in. [247.65 mm] gear— 

gashed, hobbed, then lapped to. 

Hardened, ground and lapped 
center hub (inner race) to size 
and roundness. 

4. Wormshaft bearing lapped for exact lineup in 

bearing sleeve above. 

5. Hardened, ground and lapped 3-in. 

[76.2 mm] ID center bushing (outer race) 
lapped to. 

6. Eighty lapped rollers (in bronze retainer). .. 


Tolerance 

+ 25 millionths in. [0.0006 mm] 
— 0 millionths in. [ — 0 mm] 


20 millionths in. [0.0005 mm] 
20 millionths in. [0.0005 mm] 
± 4 seconds (Precise) 

± 2 seconds (Ultra-Precise) 


5 millionths in. 

+ 25 millionths in. 
— 0 millionths in. 

± 5 millionths in. 
size 

5 millionths in. 
roundness 

millionths in. 
(identity) 


[0.000127 mm] 
[0.0006 mm] 
[-0 mm] 

[0.000127 mm] 

[0.000127 mm] 

[0.0000635 mm] 
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FIG. 355 —The rotary table year is indexed 

and gashed automatically on a special 
machine constructed from a modified 
measuring machine base. 



FIG. 35G— The rotary table gear is hobbed 
while rotating synchronously with a. 
master rotary table on which it is mounted. 
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FIG. 357 —Final accuracy of the rotary 
table gear is achieved by careful lapping 
while in its own assembly. The worm-lap 
is used only for one gear, then discarded, 
yet must be made with virtually the same 
care as the worm itself. 


Manufacture of the 

Rotary Table Worm and Gear 

Since the worm and gear represent the two 

most important elements of the rotary 

table, it is valuable to briefly examine how 

they are manufactured. 

The Gear— The gear of the rotary table 
begins as an exceptionally close-grained 
cast iron blank. The gear blank is ground 
flat and parallel and an accurate hole is 
bored in its center. The gear blank is located 
from this hole on a special machine where 
the teeth are gashed, Fig. 355. 

This single-purpose machine consists of a 
modified measuring machine base, a rotary 
table for indexing the gear automatically 
through 2° increments, and a precision 
spindle holding a gashing cutter fixed at 
the correct helix angle. 

The gear, after being gashed on the 
machine, is removed to a second machine, 
located accurately from the center hole and 
hobbed. The second machine consists es¬ 
sentially of a master rotary table which 
carries the gear, revolving synchronously 
with a spindle-mounted hobbing cutter, 
Fig. 356. The hobbing operation develops 
a true involute form on the teeth. 

Although both gashing and hobbing are 
closely controlled, the gear error still ex¬ 
ceeds specifications. Final accuracy of the 
gear is attained only after it becomes part 
of a full rotary table assembly. Once mated 
to the parts which make up the full con¬ 
struction, especially its own worm, thr-most 
accurate calibration of the gear can be 
attained. Final correction proceeds by hand 
lapping. The cast iron lap used, Fig. 357, is 
made with great care, and is assembled in 
exact alignment during corrective lapping 
to insure matching its involute form with 
that of the gear being lapped. The lap has 
a very short working life, since during lap¬ 
ping, thread form wears rapidly. The gear 
itself must be corrected with as few lapping 
operations as possible, lest its involute form 
be destroyed. 



The gear teeth are lapped by working 
always to the smallest tooth, shown sche¬ 
matically in Fig. 358. Theoretically, the 
smallest tooth is not lapped at all. The other 
teeth are lapped in increasing proportion to 
their “thickness” (or the amount they are 
“plus”). The inspection chart used by the 
lapping expert is shown in Fig. 359. The 
derivation of this chart is shown on page 
239. 

The Worm —Perfection of the worm is 
essential for overall accuracy of the rotary 
table, especially with regard to periodic 
error. Pitch diameter must be concentric 







fig. 3GS — (top) The gear teeth of the rotary 
table are lapped in proportion to their 
“thickness” as determined by calibration 
with the 1H0 Index. 


fig. 359 —( center ) The lapping specialist 
is provided with a complete graph of 
the rotary table error before each lapping 
operation. He analyzes the chart carefully 
before proceeding to lap, bearing in mind 
the results of any previous lapping 
operations. 


Not 

lapped 


Lapped 

slightly 


Lapped 

heavily 





with its own journal (support) bearing, lead 
must be precise, and the worm must be 
exactly aligned with the gear in assembly. 

Just as in thread grinding the leadscrew, 
the worm is flooded with temperature- 
controlled oil. Worm lead, however, is con¬ 
trolled by a master worm in a special fix¬ 
ture, Fig. 360. If inspection reveals errors, 
the worm is correctively lapped. 


The Importance of Accuracy of 
Rotation in a Rotary Table 


Trueness of rotation is seldom encoun¬ 
tered on any rotary table manufacturer’s 
list of accuracy specifications. Yet in al¬ 
most every rotary table application, lack 
of true spindle rotation results in both 
angular error and center-distance error. 
The nature of this error is illustrated in the 
hypothetical workpiece in Fig. 361. As¬ 
sume that the actual angle generation of 
the rotary table is perfect, but that the 
axis of rotation wanders from A to B and 
back to B for each revolution. This would 
appear approximately as in the lower right 
of the diagram if a polar recording were 
obtained by centering a round master gage 
to the spindle as in Fig. 364. 

If the rotary table had an eccentricity 
radially of 0.000050 inch (0.00125 mm), the 
center-distance of the holes would, of 
course, be in error by 0.0001 inch (0.0025 
mm). As shown in the diagram, an angular 
error might also result, amounting to a 


maximum of Sin 6 


AB 
2R' 


The angular error which results is even 
more interesting. On an 8-inch [203.2 mm] 
diameter circle, the included angular error of 
the holes from the true center point is 2.577 
seconds. If the holes had been machined 
in a pattern of 3-inch [76.2 mm] radius from 
center the error would be 3.44 seconds; at 
1 inch [25.4 mm] 10.31 seconds; at 0.100 
inch [2.54 mm] 1 minute and 43.13 seconds; 
and at 0.010 inch [.254 mm], 17 minutes 
and 11.31 seconds. 


FIG. 361 —The overall performance of the 
rotary table is greatly dependent on 
perfection of the worm. Lead and 
concentricity of each worm are controlled 
from a master worm-grinding fixture which 
mounts on the thread grinder. 
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fig. 361— A rotary table spindle that does 
not rotate “true” introduces an additional 
angular error quite apart from the inherent 
angular errors specified by the rotary table 
manufacturer. This becomes apparent by 
citing as an example a rotary table with a 


spindle axis wander of 50 millionths of an 
inch [1.2pm). Shown is an approximate 
scaled up representation of one type of 
angular spacing error which could result 
in a part. 









FIG. 302 —A plain-bearing spindle, used 
horizontally, has two axes, one while at 
rest where the oil is squeezed tut at the 
bottom; the other displaced higher, where 
the oil is evenly distributed while rotating. 




As the pattern of holes is put on an in¬ 
creasingly smaller-diameter circle, a hypo¬ 
thetical condition is approached in which 
all holes are in a straight line, or an error 
of 180°. 

It is generally known that an angular 
error decreases the effective error linearly, 
the smaller the radius. The previous ex¬ 
ample makes clear a lesser known fact. 
Runout has the effect of increasing the 
angular error, the smaller the radius. The 
angular errors which result in practice from 
rotary table eccentricity are in addition to 
its calibrated angular errors. 

Rotary tables are usually constructed 
with a small plain (metal-to-metal) bear¬ 
ing. The interfaces are separated by a thin 
lubricating oil film. This type of journal 
must allow at least 100 millionths of an 
inch [0.0025 mm] clearance to permit free 
rotation, since the oil film increases in 
thickness and stiffness as rotational speed 
increases. If a clearance of approximately 
50 millionths of an inch [0.00127 mm] or 
less is used, seizing and galling almost 
always occur. 

Trueness of rotation is absolutely depen¬ 
dent on the uniformity of the oil film. While 
it is true that a thin oil film, if confined, 
strongly resists compression, it is equally 
true that oil can roll or slide quite easily— 
either axially or circumferentially—when 
carried along between rotating or sliding 
members. If the load on the spindle is 
inconstant, as is the case when the rotary 
table is carrying a heavy unsymmetrical 
workpiece or when it is mounted on the sine 
table, the oil film tends to be thicker on one 
side. The thicker film of oil acts as a solid 
body, causing sideways displacement of the 
rotating member. 

A plain-bearing spindle, when used hori¬ 
zontally, actually has two axes—one when 
it is at rest and all the oil is squeezed out at 


fig. 303 —A typical characteristic of any 
plain-bearing spindle is for it to shift its 
axis on every other revolution. 
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FIG. 364 —A polar recording oj rotary table 
“trueness of rotation” is made by reference 
to a master ball, round to .000001 inch 
[0.000025 mm}. The rotary table spindle 
is checked with axis vertical (shown), and 
while horizontal by using its built-in 
right angle feature. 


the bottom, and another at a higher point 
when it is rotating and the oil is evenly 
distributed, Fig. 362. 

Even under ideal conditions the oil film 
is never truly uniform. It tends to build up 
and disperse in waves. Typically, the axis 
is displaced at every other revolution, Fig. 
363. 

In order to attain a trueness of rotation 
commensurate with the ± 2 seconds accu¬ 
racy of the rotary table, Moore uses a roller 
spindle, with a guaranteed rotational accu¬ 
racy of 10 millionths of an inch [0.00025 
mm] or less. The construction of this spindle 
has previously been shown in Fig. 354. The 
rollers which make up the assembly are 
held alike in size to less than 3 millionths 
of an inch [0.000076 mm] (see page 133) 
and under stiff preload between inner and 
outer race. 

Each rotary table is calibrated for true¬ 
ness of rotation in both a horizontal and a 
vertical position, Fig. 364. A circular chart 
of trueness of rotation accompanies each 
rotary table along with its calibration 
chart of angular accuracy, Fig. 365. 

Rotary tables were first developedforuse 
with milling machines and then with jig 
borers and jig grinders, and find their 
greatest number of applications in the 
toolroom and the machine shop. Typical ap¬ 
plications of this type are shown in “Uni¬ 
versal Measuring Machine Techniques 
and Applications,” pages 310-313. Today, 
rotary tables are found in the inspection 
laboratory, too. One inspection application 
is seen in Fig. 366. Here the built-in right- 
angle feature of a rotary table is being uti¬ 
lized in the inspection of a master template. 
In another inspection application, Fig. 367, 
a rotary table equipped with an accessory 
tailstock is used to inspect the angular 
spacing of the splines on a piece. The piece 
mounted between centers is 35 inches [889 
mm] in length 




FIG. 365 —A circular chart of triteness of 
rotation as well as the annular calibration 
chart of each Precise and Ultra-Precise 
Rotary Table is supplied lo the user. 
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fig. 366 —A master angle template is 
inspected while the rotary table is 
mounted at right-angles, using a surface 
plate, indicator and stand. 




5. Principal Angle 
Standards 

As shop tools, the rotary tables are em¬ 
ployed in the machining of an almost infinite 
variety of mechanical parts where angular 
relationships are to be generated to a high 
degree of accuracy. When more accurate 
rotary tables are used as metrological in¬ 
struments, they often provide the “final 
word” on angular accuracy. 

Manufacturers of rotary tables verify the 
accuracy of their products through the 
use of angle standards. The principal 
types of angle standards (suggested by the 
National Physical Laboratory, England) 
are shown in Fig. 368. An analysis of the 
angle standards that might conceivably 
be applied to the calibration of rotary 
tables follows. 

ANGLE GAGE BLOCKS 

A useful type of angle standard is the 
angle gage block, Fig. 369. This standard 
consists of a hardened-and-lapped steel 
block, generally wedge-shaped, with a spe¬ 
cific included angle between two flat work¬ 
ing surfaces. Angle gages are available in 
sets consisting of a number of blocks, each 
with a different included angle. 

The first set of angle gages was devised 
by Tomlinson of the NPL in 1939. It con¬ 
sisted of twelve blocks having included 
angles of 3, 9 and 27 seconds; 1, 3, 9 and 27 
minutes; and 1, 3, 9, 27 and 41 degrees. A 
single square block was also included in the 
set. 

Angle gage blocks were doubtless in¬ 
spired by the familiar Johansson type gage 
blocks used in length measurements. Just 
as with Johansson type gage blocks, angle 
gages can be wrung together to form fixed 
standards—in this case for angle, rather 
than for length. 

These two types of standards differ in 
other respects, however. Relative length is 


fig. 307 —An accessory tailstock feature 
converts the rotary table into a “dividing 
head” with capacities and features not 


found in most “dividing heads." The 
piece mounted between centers is 35 inches 
[<5S9 mm] in length. 
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FIG. SOS --Principal types of angle 
standards. 

Courtesy of tho NPL. British Crown Copyright. 


Principal Types of Angle Standards 


Polygon 


Angle standards of polygonal form have a 
number of equally inclined and optically flat 
facets. Polygons are manufactured having up 
to 72 facets, i.e., the smallest exterior angle 
which can be defined in practice by the adjacent 
faces of a polygon standard is 5 degrees. 



The exterior angles between adjacent 
faces of a polygon standard can be calibrated 
from first principles, with the aid of two auto¬ 
collimators, to an accuracy of ±h sec of arc. 


Serrated Type 


Indexing Devices 


Ball Type 



Different indexing positions are obtained by 
means of two matched sets of either equally spaced 
radial serrations or contacting balls of the same dia¬ 
meter arranged in a circle mounted in the fixed base 
and the upper rotatable table. Any angle corresponding 
to a number of serrations or balls can be indexed to an 
accuracy within sec of arc for the serrated type and 
±1 sec of arc for the ball type device. 

The ball type device as shown has been designed 
to operate in either the horizontal or vertical plane. 


Rotatable Table 



Base 



FIG. 369— Angle gage blocks can be 
obtained at a. moderate expense, and can be 
wrung together addilively or subtra(‘lively 
to form almost any required angle. 

Courtesy L S. StarreH Co. Webber Gage Div. 
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FUi, no~~ Hjfc.-polygm ?'s <<■ principcd 
circle-dividing standard, having a nnrnber 
of reflective surfaces on Us periphery set to 
a specified angle for use in conjunction 
with an autocollimcitor. 



not significant with angle gages—only the 
included angle effects the measurement. 
Thus angle gages can be wrung together 
additively, or be turned end-for-end, sub¬ 
tracting from angular values. For this rea¬ 
son, far fewer angle blocks are required in a 
set than is the case with a set of length¬ 
measuring blocks. Tomlinson’s set of only 
twelve blocks allowed any angle to be set, 
in increments of 134 seconds. 

While angle gage blocks have many use¬ 
ful applications, such as refining the settings 
of tilt tables and serving as angular refer¬ 
ences in surface plate inspection, the range 
of possible applications is limited. Angle 
gage blocks are not as widely used as length¬ 
measuring gage blocks. 

Angle gage blocks seem to lack the requi¬ 
sites for use as primary standards because: 
(1) errors are easily compounded when 
angle blocks are wrung in combination; (2) 


the absolute verification of angle blocks is 
usually dependent on some other primary 
standard. 

THE POLYGON 

For many years, and until quite recently, 
the polygon, Fig. 370, which is used in con¬ 
junction with an autocollimator, was con¬ 
sidered the most accurate device for the 
calibration of circle-dividing equipment, 
including the most accurate rotary tables. 

Polygons are usually no more than 6 
inches [152 mm] in diameter—small enough 
to be given full support when centered on 
the smallest rotary table (polygons with 
smaller and larger diameters are sometimes 
made as “specials”). Most polygons are 
not over 1 or 2 inches high [25-50 mm]. 
Because of this low profile, the magnifica¬ 
tion of angular errors caused by “wobble” 
is excluded as much as possible from the 
inspection of pure angle. 

The earliest model polygon, developed by 
Taylerson of the NPL* consisted of a 
hardened steel block. A series of optically 
flat reflective faces was lapped directly on 
the block. Later commercial models were 
of glass. The faces were coated to improve 
their reflectivity. Recently, the trend has 
been toward the use of separate, optically 
flat mirrors, which are individually mount¬ 
ed on the periphery of the polygon. These 
are mounted carefully to attain as exact 
angular spacing as possible. They must be 
perpendicular to the mounting surface, and 
mounted in a manner that does not create 
distortion. 

The number of flats or mirrors in a poly¬ 
gon depends on the particular division of a 
circle that is sought. To insure certainty of 
calibration, the primary consideration in 
accuracy is not how closely the mirror 
agrees with the nominal angle, but how flat 
and square it is. 

*J. C. Evans & C. O. Taylerson, “Measurement 
of Angle in Engineering,” NPL Notes on Applied 
Science, p. 6. 
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fig, 371 -Polygons ware developed for 
calibrating circle-dividing equipwenl, 
notably rotary tablet. Rotary tables are now 
calibrated most auihoritively by use of the 
Hl r O Precision Index, 


It is generally agreed that seventy-two 
mirrors (5°) is the maximum number that 
can be employed on the largest practical 
polygon (about 12 inches [304.8 mm] in 
diameter) while still maintaining adequate 
reflectivity and flatness. Usually the num¬ 
ber of divisions is quite small—six (60°), 
nine (40°), or twelve (30°) for maximum 
reflective area and accuracy. 

Calibration of the Polygon 

Accuracy of the polygon can be proved by 
first principles (see “Theory of Calibrating 
a Divided Circle,” page 207). To calibrate 
the polygon, it is mounted on a turntable 
and the angle segment being checked is 
contained between two rigidly fixed auto¬ 
collimators nulled on the nominal angle. 
The polygon can also be calibrated directly 
by using a Moore 1440 Small Angle 
Divider, as demonstrated on page 249. 

Criteria Effecting 
Polygon Accuracy 

When selecting a polygon, there are four 
major criteria: 

1. The reflecting surfaces (flats or mir¬ 
rors) should be within 4 or 5 seconds 
of the correct or nominal angle. 

2. The reflecting surfaces should be flat 
to 0.000005 in. [0.000127 mm]. 

3. If individual mirrors are used, they 
should be mounted in such a way that 
there is no distortion. 

4. The mirror position should be per¬ 
manent. There should be no possi¬ 
bility of alterations of mirror position 
from bumping or jarring, from poor 
mounting, or from secular change 
(instability of the body of the poly¬ 
gon). 



Polygons Applied to the 
Calibration of Rotary Tables 

To calibrate a rotary table, the polygon is 
mounted concentrically on the table, Fig. 
371. Theoretically, imperfect centering has 
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FIG. 372 —Moore Rotary Tables were first 
calibrated inlerJeromelricaUy. This method 
of calibration was superseded by the 
HW Precision Index. 

From “Ho!es, Contours and Surfaces." 




a negligible effect on the accuracy of calibra¬ 
tion. In practice, it is found that the small 
amount of time required for perfect centering 
is well spent. It makes it possible to avoid any 
errors caused by variables of polygon form. 

The fixed autocollimator is nulled on the 
center of the first face of the polygon. Next 
the rotary table is rotated through the 
prescribed angles, corresponding to the 
increments (number of faces) of the poly¬ 
gon. Angular deviation shown by the 
autocollimator is recorded at each angle 
sector. The calibration is valid only if a 
zero reading is obtained when returning to 
the original setting face. 

A correction factor (error of each polygon 
face from nominal) furnished by the manu¬ 
facturer is applied to the readings to find 
the actual error of the rotary table at the 
sectors calibrated. 

Limitations of Polygon for 
Calibrating Rotary Tables 

1. Calibration or correction factor— Ordi¬ 
narily, it is specified that the center of the 
mirror be used in calibration in order to 
obtain agreement with the polygon manu¬ 
facturer’s correction factors. However, 
there is no means of reference to assure the 
user that this centering is exact. 

It is not uncommon for the calibration 
factor to change considerably if the target 
area used for calibrating the rotary table 
differs slightly from that used when the 
polygon was originally calibrated. Because 
of the number of mirrors in use, each with 
its own geometrical characteristics and re¬ 
flectivity, any irregularity may alter the 
supposed calibration value of the polygon. 
Examples of irregularities are “wobble” of 
the rotary table, imperfect leveling, or read¬ 
ing the autocollimator by eye rather than 
by photoelectric readout. 

2. Limited number of angles —It is impos¬ 
sible to make a full comprehensive calibra¬ 
tion of the rotary table with one polygon 
because the angular sectors are limited to 


FIG. 373 —Diagram of the optical arrange¬ 
ment for interferometric calibration of 
rotary tables. 

From "Holes, Contours and Surfaces.” 
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a maximum of about 72 in number. Even 
a 72-sided polygon would not detect peri¬ 
odic error of a mechanical rotary table. 
Also, not all of the lines of the circular scale 
of an optical rotary table are inspected for 
error. The secondary graticule or optical 
vernier used for finer minute and second 
increments are not inspected. 

The 13-sided polygon is often used be¬ 
cause it gives a random check more likely 
to show cumulative errors of the total rotary 
table system, whether optical or mechani¬ 
cal. Its disadvantage is that it does not by 
itself differentiate between, for example, 
worm-and-gear error. More than one poly¬ 
gon is usually employed, such as one 12- 
sided and one 13-sided. 

3. Undesirability of working with calibra¬ 
tion values, which may amount to as much as 
several seconds. 

It. Overall certainty of rotary table calibra¬ 
tion based on the polygon as an angle stan¬ 
dard is not much better than one or two 
seconds, after applying the correction factor. 
Such accuracy is not always significantly 
greater than that of the rotary table it is 
intended to calibrate. 

INTERFEROMETRIC 
CALIBRATION OF THE 
ROTARY TABLE 

A system employed successfully for many 
years at Moore for the calibration of rotary 
tables made use of monochromatic light 
waves as a means of comparison. The 
method was worked out by Frederick C. 
Victory, then chief engineer, in consulta¬ 
tion with T. J. O’Donnell of the University 
of Chicago. It was an adaptation of the 
Michelson interferometer. This was the first 
industrial use of light waves for the inspec¬ 
tion of angles. A photograph of the optical 
arrangement is shown in Fig. 372. A sche¬ 
matic diagram appears in Fig. 373.* 

*Richard F. Moore and Frederick C. Victory, 
Holes, Contours and Surfaces, p. 52. 


A classic circle-dividing technique was 
employed. A pair of adjustable mirrors, 
acting as “dividers,” was mounted on the 
rotary table to be inspected. The inter¬ 
ferometer was used as an extremely sensi¬ 
tive comparator. It was arranged so that, 
as the eye scanned between two reference 
lines, the value of each fringe observed 
equaled 1 second of arc.* 

Interferometric calibration of rotary 
tables represented an advanced state of the 
art for some time. Its greatest advantage 
over the polygon was in its ability to inspect 
many different angles with one instrument, 
since the mirrors could be set to any angle. 
Secondly, as a comparator, the inter¬ 
ferometer provided a degree of sensitivity 
which was not at that time attainable by 
the autocollimator or by any other means. 
There were, however, disadvantages in its 
use: 

1. It was not a direct inspection of the 
angle. The circle first had to be 
“closed” before the value of the seg¬ 
ment set by the mirrors was deter¬ 
mined. Since it was impractical to 
close the circle with increments as 
small as 2°, “tie-down” points were 
first established at larger angle sec¬ 
tors, such as 90° and 18°, from 
which further subdivisions were 
made. Small errors could therefore 
accumulate. 

2. It was difficult to resolve worm error 
since it was not inspected directly. 

SERRATED-TOOTH 

CIRCLE-DIVIDER 

Credit is due the A. A. Gage Co. for per¬ 
fecting an old principle for application as a 
precise circle-divider. The principle is as 
follows: two “face-gears” are generated, 
each identical in diameter and in number 
and shape of teeth. When the two opposed 
faces of the gears are brought into forced 
engagement (root-to-crest), they become 
*Ibid., p. 50. 


FIG. 374 — The serrated-tooth divider 
employs two fare gears of identical shape 
and spacing of teeth. One member is 
displaced axially to disengage the teeth, 
then rotated radially to the desired 
nominal angle. The “rotating” member 


I ——j per tooth 



need only be approximately located over 
its mate (by reference to scale and vernier 
on its periphery) when re-engaged because 
of the “homing” characteristics of the 
multiple teeth. 
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FIG. 375 —The 1 ! fLO Precision Index 
allows IffO awjl.es (every Vi°) within a 
circle to he set to ± 0. t second of are 
accuracy. 



locked in place, preventing rotation or side 
movement. When the teeth are disengaged, 
rotated and brought into contact once 
again, the angular displacement corre¬ 
sponds to the number of teeth through 
which it has been indexed, Fig. 374. 

High precision comes about in two ways: 

1. The averaging effect of all the teeth 
in meshed contact generates an index¬ 
ing accuracy far surpassing that exist¬ 
ing in the angular spacing of indi¬ 
vidual teeth. 

2. When the teeth are lapped in ran¬ 
dom contact, even greater accuracy 
through averaging takes place, since 
the tendency is for all the teeth to 
conform in size, shape and pitch (the 
“fat” teeth are lapped heavily; 
“thin” teeth are lapped lightly). 

Moore 1440 Precision Index, 

Fig. 375 

The “serrated tooth circle-divider,” used 
in conjunction with an autocollimator, 
appeared to have tremendous potential 
when applied to the calibration of rotary 
tables. However, the existing serrated tooth 
circle dividers were limited to 360 teeth. 
This number of teeth was deemed inade¬ 
quate for a comprehensive rotary table 
calibration. 

It was decided to develop a serrated 
tooth indexer* in 1/4° increments having 
1440 teeth, (4 x 360). Since Moore Rotary 
Tables have 180 teeth, this would allow an 
eight point check of the worm. An 8-inch 
[203 mm] diameter was selected as the best 
compromise for the size of the 1440 indexer. 
When of this diameter, it was small enough 
to fit on any rotary table, yet large enough 
to support small parts and gages. 

Technically, it was feasible to put 1440 
notches around a 24-inch [610 mm] cir¬ 
cumference. Practically speaking, however, 
engineering difficulties were encountered in 

*A. A. Gage Co., Detroit, Michigan, are holders 
of the serrated tooth patent. 


fig. 370 . -To attain the ±0.1 second 

accuracy of the 1 hW Index, the teeth of 

each member must be ground as 
accurately as possible, prior to lapping. 

At this stage , a measuring machine ha.se is 


used for controlled geometric movement. 
Angular spacing is established by the 
master indexer on which the face gear is 
mounted while grinding the serrations. 
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accomplishing this with anything approach¬ 
ing perfection because of the small tooth 
size. Tooth-to-tooth spacing with this de¬ 
sign was only 0.017 in. [0.432 mm], allow¬ 
ing only a few thousandths of an inch 
[hundredths of a mm] contact on each 
tooth flank. 

Tooth geometry and tooth spacing would 
have to be exactly maintained over all 
1440 teeth for the device to perform to 
expectation. For example, radii on the root 
and crest of the teeth could not exceed 
0.001 in. [0.0254 mm] to achieve flank con¬ 
tact of the teeth and avoid “bottoming.” 

The goal was an accuracy fine enough that 
no calibration factor need be applied when 
calibrating rotary tables, and one that would 
also satisfy the “gage-maker’s 10 to 1 rule”— 
namely ±0.1 second accuracy. 

Within 6 months of research, a 1440 
Index was built to ± 0.3 second accuracy. 
It was evident from the experience gained 
that the principles employed to produce 
the 1440 Index were correct, but that the 
sought-after ± 0.1 second accuracy would 
only be achieved by even more closely re¬ 
fining each element of the manufacturing 
process from start to finish. 

A special grinding machine was con¬ 
structed, using a measuring machine base, 
Fig. 376. A special-purpose grinding wheel 
headstock and dresser were incorporated 
into the modified column. To attain the 
utmost uniformity and accuracy, all cycles, 
including rotary indexing, were made com¬ 
pletely automatic and free of operator 
variables (it had been found that the gen¬ 
eration of 1440 teeth by manual operation 
was beyond the endurance of the machine 
operator). 

The grinding wheel itself was an object 
of prolonged study. It is of a specific grade 
that will maintain free-cutting qualities, 
yet keep its size and shape without re¬ 
dressing over the final one thousand, four 
hundred and forty passes. 

Very close tolerances of roundness, 


cylindricity and concentricity of all sur¬ 
faces must be maintained. A Spin Table 
(see “Roundness,” page 259), was under 
development at the same time, and had 
just been perfected. Its use, together with 
the jig grinder, enabled the roundness and 
flatness of certain surfaces of the 1440 
Index to be refined to an accuracy pre¬ 
viously unattainable. This is a further 
example of how the progress of one me¬ 
chanical art depends on the refinement of 
another. 

Stability, Heat Treatment 

It is sometimes presumed that a ser¬ 
rated tooth circle-divider, once completed, 
will continue to improve in use because of 
the continuing averaging effect of wear. 
There is no denying that the nature of its 
operation tends to improve accuracy. How¬ 
ever, the amount of controlled wear given 
it during lapping so far exceeds that which 
would occur during normal use that the 
validity of this claim is doubted. 

The 1440 Index is made to engage and 
disengage automatically for hours at a 
time. At first, a lapping compound with 
relatively coarse grit is used. As the teeth 
match closer and closer, lapping compound 
with progressively finer grit is employed. 
At certain strategic stages, the lapping 
operation is interrupted and the OD’s of 
the teeth and secondary reliefs are re¬ 
ground. Finally, all lapping compound is 
flushed out and only kerosene is used. At 
the very last, the mating sections are made 
to engage and disengage for hours while 
completely dry and clean to achieve perfect 
engagement, Figs. 377 and 378. 

Exactly how long the 1440 Index can 
maintain its accuracy is unknown at pres¬ 
ent. Many more years of experience will 
be required. It is doubtful that wear of the 
teeth will either significantly improve or 
reduce accuracy. More than likely, any 
change will be caused by metallurgical 
instability, not by wear. 







)•!<:. :jry -Timt / J ’p) Indexes must be 
employed to calibrate the 141 +Q Index. 

The classical circle-closing technique used 
is identical to that employed by the 
world’s leading standards bureaus. 



In view of this, all possible precautions 
are taken in the heat-treatment of the 
parts to insure long-range stability. The 
method of normalizing the mating sections 
of the 1440 Index does not differ appreci- 
bly from the stabilizing process Johansson 
used to put his gage blocks “to sleep” 
nearly a century ago. 

Harden; 

Draw to 375°; 

Freeze to — 60°F.; 

Heat to 340°F.; 

Let cool to room temperature. 

This heat-and-freeze cycle is repeated 
three times, the last being the heat cycle. 

Calibration of the 11+1+0 Index 
The 1440 Precision Index is guaranteed to 
be accurate to within ±0.1 second of arc 
at all 1440 indexed positions. One might 
at first assume that to assure l/10th- 
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FIG. 3cSu — A typical matrix of readings 
obtained when one 11+1+0 Index is compared 
to another. U+J+ separate, readings are taken 
to allow calibration of Vi indexed 

positions. 
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second accuracy and still conform to the 
gage maker’s “10-to-l rule” the 1440 
Index would have to be inspected with 
some other master accurate to l/100th 
second. This is impossible, of course, since 
no such master exists. Nonetheless three 
factors enable such a guarantee to be 
stated legitimately; 

1. The 1440 Index will repeat to at least 
l/50th of a second. 

2. Present technique allows the employ¬ 
ment of monochromatic autocolli¬ 
mators, which are sensitive enough to 
detect differences as small as 1/100th 
second and repeatable to about the 
same order of accuracy. 

3. The 1440 Index can be calibrated by 
self-proving (circle-closing) methods. 

The method of self-calibration employs 
two 1440 Indexes, Fig. 379. One is mounted 
concentrically on the table of the other. On 


the uppermost 1440 Index is placed a single 
mirror of optical quality, which is shielded 
from temperature variations within a pro¬ 
tective housing. The mirror is used in con¬ 
junction with a monochromatic, photoelec¬ 
tric autocollimator. The autocollimator and 
mirror housing are close-coupled, with a 
minimum air gap to avoid variations which 
may arise from air currents passing between 
them. 

By rotating one of the 1440 Indexes clock¬ 
wise and the other counterclockwise, each 
through the same nominal sector of arc, 
the mirror is returned essentially normal 
to the autocollimator. The slight angular 
change in the attitude of the single mir¬ 
ror, as recorded by the autocollimator, 
represents the difference between the 
particular angular segments of each 1440 
Index which have been selected for com¬ 
parison. 


From a matrix of readings obtained in 
this manner, Fig. 380, (the circle also hav¬ 
ing been “closed”), both 1440 Indexes are 
self-calibrated. The reader is referred to 
Fig. 333 for the theory of self-proving the 
division of the circle. A typical calibration 
certificate is shown in Fig. 381. The cali¬ 
bration theory and method just described 
are identical to that employed by the NPL, 
the NBS and the PTB in their own calibra¬ 
tion of the 1440 Index, and are sanctioned 
by them. The results of two such calibra¬ 
tions are shown in a report by the NPL, 
Fig. 382, and by the NBS, Fig. 383. 

Rotary Table Calibration 
with the llplpO Index 

Fig. 384 shows a 1440 Index mounted on a 
rotary table for calibration. A single 
shielded reflecting mirror and autocollima¬ 
tor, much the same as that employed for 
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FIG. 381 — The standard “Certificate of 
Accuracy” of the. I I/O Index, furnished 
to the user. 




































pig. 382- -A report by the National 
Physical Laboratory on ike calibration of 
two indexing tables. 



NATJONAI PHYSICAL LABORATORY 
Trddmgton. Middlesex. England 

REPORT 

THE CALIBRATION OF TWO PRECISION ANGULAR INDEXING TABLES 


SAMOA A I, PHYSICAL 1. \ MORATORY 

OF RETORT 

CAIiIUKAIIBN OF Tt 0 PRECISION AI.LULAK IN LEUNG TABLES 


TABLE 


Made and Sent by:- Moore Special Tool Co., Inc. 

Bridgeport, Conn. 06607. U.S.A. 

Description 

Two precision 8-inch diameter angular indexing tables. The angular indexing 
of each table is effected by engaging two serrated plates, one being inte¬ 
gral with the rotatable (upper) member and the other integral with the base 
of the table. One table has 2160 serrations, enabling angular increments of 
10 minutes of arc to be indexed, whilst the other table has 1440 serrations 
enabling angular increments of IS minutes of arc to be indexed. An exterior 
degree scale and a vernier scale are provided on each table to enable the 
appropriate indexing position to be located prior to engaging the serrations. 

The 2160 serrated table has a hydraulic system for raising and lowering the 
top rotatable member whilst that of the 1440 serrated table is controlled by 
means of a cam manually operated by a lever. 

To facilitate the calibration of each table the manufacturer submitted them 
with the 2160 serrated table accurately mounted on top of the 1440 serrated 
table. 

Identification :- 

The 2160 serrated table 8 - 1440 - 10 
The 1440 serrated table 8 - 1440 - 3 

These two precision angular indexing tables have been calibrated at the 
National Physical Laboratory in the following manner. 

To minimise the effect of temperature variations during the calibration 
the mounted tables were placed inside a fabricated enclosure, and the 
necessary controls for indexing the two tables were arranged to be operated 
from outside this enclosure. 

A quartz reflector, with a reflecting face flat to within 0.000 001 in 
over its IS in diameter, was mounted on the working surface of the top table. 
This reflector was observed with a Hilger TA 5 Monochromatic Autocollimator. 

Each table was then calibrated at intervals of 22S degrees with respect 
to its zero position by comparing every 22S degrees interval of the top 
(2160) table with each 22S degrees interval of the bottom table. The results 
of these 2S6 comparisons were formed into a matrix and the angular indexing 
errors of each table at intervals of 22S degrees were calculated: these 
errors, expressed to the nearest 0.05 second of arc are given m the following 
Table. 


Indexing error Indexing error 

Table Setting 2160 serrated table 1440 serrated table 




second of arc 


second 


of arc 


0 

22h 

45 

671 , 

90 

112*2 

135 

1571, 

180 

202*5 

225 

247*5 

270 

292*5 

315 

337*2 


0 

0 

+0.05 

+ 0.10 

+0.05 

0 

-0.05 

-0.05 

-0.05 

-0.05 

-0.05 

0 

-0.05 

0 

0 

0 


0 

0 

0 

- 0.10 

- 0.10 

-0.05 

0 

+ 0.10 
+ 0.10 
♦ 0.05 
+0.05 
+0.05 
+0.05 
0 
0 

-0.05 


It is estimated that the accuracy of determination of 
the errors quoted above is 0.2 second of arc. 


29th October, 1965 
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U.S. Department of Commerce L7101-B 

NATIONAL BUREAU OF STANDARDS 
WASHINGTON, D.C. 20234 

NATIONAL BUREAU OF STANDARDS 

REPORT OF CALIBRATION 

for 


One Angular Indexing Head - 2 - 

Serial No. 8-1440-5 


Index 

ta e lg 

degrees 


Deviation from 
Nominal Angle 
seconds 


One Angular Indexing Head 
Moore Serial Nuniber 
8—1440—S 


submitted by 

The Moore Special Tool Co., Inc. 
800 Union Avenue 
Bridgeport, Conn. 06607 


0-168 3/4 e.09 
0-180 + .07 
0-191 1/4 +.04 
0-202 1/2 00 
0-210 -.03 
0-213 3/4 -.02 
0-225 00 
0-236 l/*» +.01 
0-240 00 


The deviations from nominal indexing angles for the table are 
listed in the following table. The 2 ero index mark from which 
the angles are indexed is the one on the opposite side of the 
table from the hydraulic lift hose connection. 


Index 

Angle 


degrees 


Deviation from 
Nominal Angle 
seconds 


0-247 1/2 
0-258 3/4 
0-270 
0-281 1/4 
0-292 1/2 
0-300 
0-303 3/4 
0-315 
0-326 1/4 


t. 05 
+.07 


+ .10 
♦ .11 
♦. 07 


0-11 1/4 
0-22 1/2 
0-30 


-.03 

-.04 


.00 


0-330 +.01 
0-337 1/2 +.05 
0-348 3/4 +.01 


0-33 3/4 
0-45 
0-56 1/4 
0-60 

0-67 1/2 
0-78 3/4 

0-90 

0-101 1/4 
0-112 1/2 
0-120 
0-123 3/4 
0-135 


-.03 
-.03 
-.05 
-.05 
-.05 
-.05 
.00 
♦ .02 
♦ .06 


♦ .02 

+ .07 


♦.10 


A positive (♦■) sign indicates that the indexed angle is larger 
than nominal. 

It is estimated that the listed values are accurate to 0.3 seconds. 


For the Director, 

;) // iZL*. 




G. Strang, Acting Chief 
Engineering Metrology Section 
Metrology Division 


0-146 1/4 
0-150 
0-157 1/2 


Order No. C19992 
Test No. 212.22/181886 
October 26, 1964 


FIG. 383 — A. calibration report of IJjJpO 
index No. 5 by the National Bureau of 
Standards. 








PIG. 384 --The 1J,4.0 index con be considered 
a 1140-fa ceded polygon for iifie In 
inspecting rotary tables. Advantages over 
the 'polygon for rotary table calibration arc: 

(1) ike use of a single mirror 

(2) greater number of indexed positions 

( 3 ) far greater accuracy. 


self-calibration of the 1440 Index, are used. 
The arrangement is typical of that already 
supplied by Moore to rotary table manu¬ 
facturers and to the leading national 
bureaus for inspection of rotary tables. 

In this arrangement, the standard cam- 
operated disengagement of the 1440 Index 
teeth has been replaced by a hydraulic lift. 
The hydraulic lift performs two functions 
— disengagement and reengagement of the 
two halves of the 1440 Index during op¬ 
eration, and simultaneous feeding of a 
plunger into a V-type indent in the upper 
member of the 1440 Index carrying the 
single mirror. 

Once its upper member is restrained, the 
lower member of the 1440 Index, being 
clamped to the rotary table, rotates in¬ 
tegrally with the table as it is cranked 
to position. When the hydraulic lever is 
retracted, the upper member of the 1440 
Index carrying the mirror floats down into 
mesh once again with its mate. The re¬ 
straining plunger is also automatically re¬ 
tracted lest it influence the teeth as they 
engage. The position of the upper member 
relative to its base must only be approxi¬ 
mate prior to being re-engaged because of 
the “homing” characteristic of the multi¬ 
ple teeth. 

Errors of the rotary table are read 
directly on the autocollimator, requiring no 
“closing of the circle,” nor use of calibra¬ 
tion factors. Errors of the 1440 Index in 
this application may be considered neg¬ 
ligible. 

Fig. 385 provides an overall view of a 
unique rotary table calibration facility,* 
recently perfected. The principle of its 
operation, seen in Fig. 384, is based on the 
1440 Index, except that all operations have 
been refined and automated, and the errors 
automatically charted. A close-up view of 
the mechanics of the system is shown in 


*Albert E. Johnson, “Incremental Calibration of 
Rotary Tables goes Automatic,” Quality Assurance, 
July, 1969. 








fig. 3SG— - Close-up of mechanics of the 
automatic rotary table calibration. The 
rotary table, to 'which is mounted a large 
gear, is driven by a swing-away stepping 
motor. A small probe retains the upper 
member of the mo Index after being 


disengaged hydraulically; the lower half is 
fixed to rotate with the rotary table. The 
restraining probe is retracted prior to the 
upper half of the Index descending 
into re-engagement. 


FTG. 387— The essential elements comprising 
he automatic rotary tabic calibrator. 




This is what the apparatus looks like in close up. Mounting 
platform “A” accommodates the rotary table. Large gear “B” 
is mounted on a precise hub which concentrically locates it on 
the worm shaft. Swing-away stepping motor and pinion “C” 
can be adjusted to mesh properly. The 1440 index “D” is 
clamped to the top of the rotary table. Mirror housing “E” is 
totally enclosed except for a slight gap to allow rotation be¬ 
tween itself and the automatic position sensing autocollimator 
“F”. If the gap exceeds Vs in., ordinary air currents would 
cause the collimator output to wander. Small hydraulically ac¬ 
tuated probe “G” keys the upper half of the 1440 to an approx¬ 
imate null while it is axially separated from the lower portion. 
The probe is retracted when the 1440 is lowered into engage¬ 
ment, allowing its teeth to mesh without outside influence. 
Electric solenoid valve “H” supplies power to a hydraulic pis¬ 
ton “J” mounted below. This enables the operation of the 1440 
remotely from the programmed microswitch. 
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Fig. 386 and a diagrammatic explanation 
is found in Fig. 387. Manual operation of 
the rotary table by index pins has been 
replaced by a programmed stepping motor, 
adapted by means of a non-influencing 
coupling only for the duration of the cali¬ 
bration. The responses of the photoelec¬ 
tric autocollimator are converted into an 
electric signal which is amplified and re¬ 
corded on a linear graph. 

Calibration may be programmed for 
increments of 2° (180 positions), 1° (360 
positions), Y 2 (720 positions), or ) 4° 
(1440 positions). Fig. 388 shows a calibra¬ 
tion chart being made of an Ultra-Precise 
Rotary Table in Y\ increments, over 1440 
individual positions. Worm-and-gear error 
are clearly defined. Time elapsed for this 
total calibration was only 80 minutes. It is 
not conceivable that a calibration this 
accurate and this comprehensive could be 
achieved by any other means. 

The automatic inspection of rotary 
tables has relieved the lapping specialist of 
a great burden. Previously, many hours 
were required for inspection following each 
lapping operation. There was much less 
certainty of total rotary table condition at 
each stage in the refinement. 

Advantage of the lkkO Index 
for Rotary Table Calibration 

The 1440 Index is undoubtedly the pre¬ 
ferred method for the calibration of rotary 
tables. The most important advantages, 
compared to other methods of calibration, 
can be summarized as follows: 

Compared to the Polygon: 

1. Cost savings* One self contained 


*Albert E. Johnson, “The Reference Circle Di¬ 
vider,” Paper presented at AO A Standards and 
Metrology Meeting, Florida, April 1966. 


unit is the equivalent of the following 
35 polygons: 


1440 

240 

96 

48 

30 

15 

6 

720 

180 

90 

45 

24 

12 

5 

480 

160 

80 

40 

20 

10 

4 

360 

144 

72 

36 

18 

9 

3 

288 

120 

60 

32 

16 

8 

2 


2. Far greater accuracy. At best, the 
polygon can be relied on to 1 or 2 
seconds, compared to the ±0.1 sec¬ 
ond accuracy of the 1440 Index. 

3. The absence of a calibration factor with 
which to contend. Miscalculations, 
notably from transposing signs, are 
avoided. 

4. Smaller angular increments can be cali¬ 
brated. Also, combined errors are shown 
(most “plus” to most “minus”). 
The ability to discriminate angular 
errors of Y\° arc allows the determi¬ 
nation of the source of error in the 
rotary table. For example, when cali¬ 
brating an optical rotary table, the 
errors attributable to the main cir¬ 
cular scale are separated from those 
of the graticule or optical vernier. 
When calibrating a mechanical rotary 
table, worm error is easily differen¬ 
tiated from gear error. A rotary table 
having a 180-tooth gear (2° per revo¬ 
lution) is inspected at eight points of 
the worm. 

5. Only one mirror is used. The im¬ 
portance of the single mirror cannot 
be overemphasized. A large flat mir¬ 
ror may be perfected for maximum 
reflectivity. Effort need only be ex¬ 
pended to assure that it is absolutely 
flat without having to consider simul¬ 
taneously its angular position. The 
same spot in the mirror is always 
used, instead of shifting from mirror 
to mirror. The mirror actually be¬ 
comes of secondary importance. 



fig. 3SS— Idle chart flowing from the 
automatic rotary table calibrator provides 
an unmistakable, comprehensive picture 
of rotary table error f or purposes 
of corrective lapping. 
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fig. .390 —The 14/j.O index is mounded on a 
special angle, iron to make possible surface 
plate inspection of a master notched index 
plate. 


Compared to the Interferometer, the 1440 
Index has the following advantages: 

1. Errors not accumulated; 

2. There is greater resolution of periodic 
errors; 

3. Simplicity, speed, compactness; 

4. Any angle can be inspected directly; 

5. Greater accuracy; 

6. No need to compute error upon “clos¬ 
ing.” 

The 1440 Index, although originally de¬ 
signed to calibrate circle-dividing instru¬ 
ments, has also been found suitable for 
other laboratory type applications. Exam¬ 
ples are the calibration of a 30° angle gage 
block, Fig. 389, and of a master notched 
indexing plate, Fig. 390. Additional appli¬ 
cations are possible when the 1440 Index is 
fitted with the Small Angle Divider, Fig. 
391. These are shown in “Universal Mea¬ 
suring Machine Techniques and Applica¬ 
tions,” pages 315-316. 



6. The 1440 Small Angle 
Divider 

An obvious limitation of the 1440 Index is 
in not being able to divide the circle into 
any number of parts other than the factors 
contained in the number 1440. For exam¬ 
ple, it cannot divide the circle into thirteen 
parts. 

The integration of an auxiliary small 
angle dividing facility to the 1440 Index 
allows any 0.1 second increment to be gen¬ 
erated and read directly, or to 12,960,000 
parts in a circle. The nature of the design is 
such that the 0.1 second accuracy of the 
1440 Index is not altered at its indexed 
positions in any way. 

The accumulated error of the small angle 
facility and the 1440 Index working to¬ 
gether as an integral assembly does not 
exceed ± 0.5 second of arc for the full circle. 



fig. 391 —The llj-LO Small Angle Divider 
provides all the capabilities of the IHO 
Index, hit will also set to any angle in 0.1 
second of arc increments. It has turn 
intended uses: (1) In the laboratory for the 


inspection of standards and. gages (2) for 
use with a Universal Measuring Machine 
where it will perform most of the functions 
of a rotary table but to much greater 
accuracies. 
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fig. 392 —The y^oth inch pitch 
[approximately 5 threads/mm ] micrometer 
screw of the Small Angle Divider produces 
extremely high resolution of angle. 
Thread-size proportions resemble that which 
ivould be obtained by winding a human hair 
around a % inch [15.875 mm] cylinder. 



DESIGN PRINCIPLES 


In the light of the analysis made in the 
early part of this section, it is interesting to 
note that all the principles employed in 
generating the small angle are essentially 
functions of length measurement; also that 
the relation of length to angle finally de¬ 
pends on a master 34° segment which has 
been established by dividing the circle. 

The Small Angle Divider feature is con¬ 
structed as an integral submember of the 
1440 Index, held true radially and axially 
by precision rollers under preload. It has a 


common axis of rotation with the 1440 
Index. 

High magnification of movement (ap¬ 
proximately 1400 to 1) is achieved through 
use of a l/120th inch pitch [approximately 
5 threads per millimeter] micrometer screw, 
Fig. 392. Adjustable, double micrometer 
nuts hold backlash to less than 0.2 second. 
Setting accuracy is enhanced by an accu¬ 
rately graduated, 4-inch [100 mm] dial 
which stays flush to a sliding vernier. 

The lapped-flat terminal face of the 
micrometer screw thrusts against a small 
carbide wheel fastened to an arm, which is 
in turn fixed to the base of the 1440 Index. 
Contact of the wheel with the anvil of the 
micrometer screw is maintained by a con¬ 
stant-pressure spring. 

The carbide wheel is nominally 5.72 
inches [145.3 mm] from the center of rota¬ 
tion. By altering the thickness of a spacer, 
the distance of the wheel from center is ad¬ 
justed slightly until 34° movement—as 
read by the Small Angle Divider—is ex¬ 
actly equal to any one of the 34° move¬ 
ments of the 1440 Index (a built-in master 
]4° segment). The wheel actually travels 
only y$° on either side of the tangent point 
of the micrometer on this 5.72-inch [145.3 
mm] arc. A diagram of the system is shown 
in Fig. 393. 

The ability of the 1440 Index to be set in 
increments as small as 34° is now seen as an 
immense benefit. When the 1440 Index is 
used in conjunction with the Small Angle 
Divider, the difference between the length 
of an arc of 34° at 5.72-inch radius [145.3 
mm] and a straight line at that radius, is neg¬ 
ligible (0.012479104 inch vs. 0.012479094 
inch) [0.31696924 mm vs. 0.31696899 
mm]. In angular values, the difference is 
only 0.000359 second (the angular varia¬ 
tion caused by the sliding of the wheel 
against the thrust face of the micrometer 
anvil, being even more negligible, was left 
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pig. 393 -. The Small Angle Divider 

assembly is a sub-member, and rotates 
the JUO Index through any angle within 
15 minutes of arc to an accuracy of 0.1 to 
0.2 second. The principle of its operation 
is illustrated. 


out of these calculations). Micrometer 
readings may be taken directly; no calcula¬ 
tions are necessary, Fig. 394. 

Setting an Angle with the 
1440 Small Angle Divider 

One revolution of the micrometer screw of 
the 1440 Small Angle Divider equals 5 
minutes of arc, three turns being equiva¬ 
lent to exactly 15 minutes ( 34 °) of arc. 
Each full turn, or 5 minutes, is recorded by 
a small pointer against a white dot on the 
barrel of the micrometer. Minutes and 
seconds are read on the dial and tenths of a 
second on the vernier. 

As an example of setting to a particular 
angle, suppose that a 13-sided polygon is to 
be calibrated using the 1440 Small Angle 
Divider, the first angle of 13 is: 

1/13 x 360° = 27° 41' 32.3" 

In this case, the 1440 Index is disengaged 
and rotated from 0° to 27° 30' as read on 
its side reference graduations and dropped 
into position. The Small Angle Divider is 
rotated from 0° to If' 32.3". 

Contrast this design and setting con¬ 
venience with a similar device having only 
360 indexed positions and in which gage 
blocks are used to generate the smaller 
angles within 1°. The difference in length 
between an arc of 1° and a straight line at 
that radius is no longer negligible. There¬ 
fore, the required combination of gage 
blocks would have to be calculated with 
trigonometric tables. To set to increments 
of 0.1 second to an accuracy of 0.2 or 0.3 
second would require the availability of 
gage blocks with increments as fine as 1 
millionth of an inch [0.000025 mm], and 
the correct sizes to enable 36,000 different 
combinations to be selected. 




= 0.0000136047 


AX = 7.5 min. 

AS —7,499994 

DA, angular error= 0.0000059905895 min. or 
0.00035943537 sec. 
Changing R= 5.72 by DR = 0.0000136047 
New R = 5.7199864 


X = Linear movement of micrometer screw 
S = Intended arc length 
AX = Intended angle 
AS = Actual angle 

DA = Difference in angle of AX and AS 

0.012479104, length of arc 
0.012479094 , length of straight line 
0.000000010, difference at 5.72 radius 


X = 0.012479064 
S = 0.0124790739 

Diff = 0.0000000099 Linear measurements in inches 


FIG. 394— The Small Angle Divider is 
fixed tangent to the-lift) Index at a 
nominal radius of 5.72 in. [ 7 / 5 .3 mm\. 
At this radius, the difference in length of 
a n arc of i,% a and a straight line is 
1 /100 of a millionth of an inch 
\1 /4.000 pm), or 0.00036 seconds of 
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METHOD OF CALIBRATION 
OF THE 1440 SMALL ANGLE 
DIVIDER 

Calibration of Overall 15 minutes 

Overall travel of the micrometer screw is 
easily self-calibrated against any one of the 
14° increments of the 1440 Index, using an 
autocollimator and a mirror mounted on 
the table. First the autocollimator is nulled. 
The Small Angle Divider is made to gen¬ 
erate an angle of exactly 15 minutes. The 
1440 Index sets to 14° of an arc in the op¬ 
posite direction, returning the mirror nom¬ 
inally to the original setting. Error of the 
Small Angle Divider is read directly with 
the autocollimator. 

Millionth Micrometer 
To calibrate the Small Angle Divider with¬ 
in the 15 minutes was a much more difficult 
task, since there was no device available to 
match its accuracy. 

Inspections were made by comparing the 
Small Angle Divider to the lead of a Uni¬ 
versal Measuring Machine. One 1440 Small 
Angle Divider was actually transported to 
the NPL in England for calibration against 
an autocollimator of known error. 

Considering the accuracy of these 
methods, they only seemed to verify that 
the Small Angle Divider, in conjunction 
with the 1440 Index, could work safely 
within an overall accuracy of +0.5 second; 
but they did not actually calibrate the Small 
Angle Divider. 

A special, horizontally-operating “Mil¬ 
lionth Micrometer” capable of calibrating 
the Small Angle Divider, Fig. 395, was 
finally designed and constructed. This 
device consists of a 1-inch [25.4 mm] OD, 
100 threads-per-inch [approximately 4 


threads per millimeter] lead screw having 
double-adjustable nuts and a frictionless 
indicator carriage. All parts were hardened, 
ground, lapped and stabilized. The 712- 
inch [190.5 mm] micrometer dial and 
vernier enable direct readings to 1 millionth 
of an inch [0.000025 mm]. 

The Millionth Micrometer itself has 
been calibrated against a series of 26 gage 
blocks of varying lengths. They had pre¬ 
viously been sent to four leading gage 
laboratories, and also to the National 
Bureau of Standards for calibration. 

Although disagreement among the four 
gage laboratories on the length of the gage 
blocks was a surprising 5 millionths of an 
inch [0.000125 mm], by discarding the 
extremes, certainty of the gage block 
lengths was felt to be approximately ± 1 
millionth of an inch [0.000025 mm]. 

Error of the Millionth Micrometer, 
calibrated by the gage blocks, appeared 
not to exceed 3 or 4 millionths of an inch 
[approximately 1/10 /um], including peri¬ 
odic error over its full one inch [25.4 mm] 
of travel. 

Since the Millionth Micrometer is used 
for a maximum of 0.050 inch [1.27 mm] 
travel for calibration of the Small Angle 
Divider, the discrepancies introduced by 
lead error of the former can be considered 
negligible. 

Calibration Technique —Fig. 396 shows 
the method and Fig. 397 offers a schematic 
of the arrangement used to calibrate the 
1440 Small Angle Divider over 15 minutes 
of its travel. 

A long tangent arm is affixed to the 
table of the 1440 Small Angle Divider. The 


Millionth Micrometer carries an electronic 
indicator reading to 1 millionth of an inch 
[0.000025 mm], which is positioned to gage 
at right angles to a lapped-flat surface on 
the tangent arm. 

The Millionth Micrometer is positioned 
to gage on this tangent-arm surface at 
a nominal radius of 11.458 inches 
[291.03 mm] from the center of rotation 
of the 1440 Small Angle Divider. This is 
just twice the radius of the thrusting point 
of the Small Angle Divider l/120th inch 
pitch [approximately 5 threads per milli¬ 
meter] micrometer screw. 

The Millionth Micrometer is physically 
moved to such a radius that five turns, or 
0.050 inch [1.27 mm], equals a 44° step 
of the 1440 Index. 

The Small Angle Divider l/120th inch 
pitch [approximately 5 threads per milli¬ 
meter] micrometer screw and the Millionth 
Micrometer screw, both set to the same 
master 14° increment, are now equal to 
one another overall, in a ratio of 5 to 3. In 
other words, five turns of the Millionth 
Micrometer equals three turns of the Small 
Angle Divider. Each full turn of 0.010 inch 
[0.254 mm] of the Millionth Micrometer 
equals 3 minutes on the Small Angle 
Divider. This relationship is ideal, since 
the Small Angle Divider can be calibrated 
in five increments (at 3-6-9-12-15 minutes) 
for lead as well as periodic error without 
introducing the negligible periodic errors of 
the Millionth Micrometer. 

In practice, however, the Small Angle 
Divider is calibrated at every 36 seconds 
of arc to obtain a more complete calibra¬ 
tion over the 15 minute movement, cor¬ 
responding to 0.002 in. [0.0508 mm] travel 
of the Millionth Micrometer. A calibration 
certificate is shown in Fig. 398. 
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FIG. son --The “M illUndh M icrometer" 
used to calibrate the SinaU Angle- Divider 
has a /ioofh inch pilch [approximately k 

FKt. 305 - When fixed tangentially at the 
correct radius, the M illionth Micrometer 
calibrates the full 15 minutes of arc 
generated by the Small Angle. Divider, 


threads/mm j micrometer screw, it ■will- 
read directly to 0.000001 inch 
1 0.000025 mm\. Circulating oil maintains 
the temperature of the micrometer screw. 





jMoore Special Oool Cojnc, 

Certificate of Accuracy 

MOORE 1440 SMALL ANGLE DIVIDER 


.»I. • »rtii> 4.M c.ta.n«M .. 

CALIBRATION CHART 




Ail. 


f_.Hbr.Uoii In 


Th' niUllonilt 


l)»|r nl CalibnflMI Apr .2? , 1^6 9 t.-Ji.'r.l-li StjAy-S. D . if.'-/. t- Sa*.*. 

A«w«*«I 'f nlpecffiV***' ’— CnrtdMir >• ....J.rl.4.4.SrJ-7sA 


fig. iw—.Arrangement of the Millionth 
Micrometer as used to calibrate the Small 
Angle Divider. 


fig. 308 —Calibration Certificate of the 
Small Angle Divider, furnished to the 
user as well as the certificate shown in 
Fig. 381. 
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fig. .39!) —The H40 Small Angle Divider 
is furnished tvith all necessary 
equipment, including a mahogany box. 




USE OF THE 1440 
SMALL ANGLE DIVIDER 

The unique construction of the 1440 Small 
Angle Divider makes a dual usage possible. 
First, it is small enough to be mounted on 
rotary tables as a calibrating standard 
with the capacity to divide the circle into 
virtually an infinite number of parts. 
Second, the fact that the small angle 
dividing feature is a submember, rather 
than being mounted on the table of the 
1440 Index (the case in a first Moore 
prototype design), leaves the table clear 
for the mounting of parts and gages. It 
will thus perform all the functions of a 
rotary table for inspection, but to many 
times greater accuracy. 

Because of the dual functions of the 1440 
Small Angle Divider, it is offered in two 
different basic sets of equipment. It is 
furnished in a mahogany box for porta¬ 
bility when used, for example, with a 
Universal Measuring Machine, Fig. 399. 
Also, it is available with a granite plate, 
mahogany cabinet and plexiglass cover 
when used essentially for calibration pur¬ 
poses, Fig. 400. Considering the special 
nature and cost of the 1440 Small Angle 
Divider, the greater protection and con¬ 
venience offered by the mahogany cabinet 
makes it preferable. 

Examples of practical applications of the 
1440 Small Angle Divider when mounted 
on a surface plate are shown here. Calibra¬ 
tions are performed on a “tilting” rotary 
table, Fig. 401, on an autocollimator, 
Fig. 402, and on a 13-sided polygon, Fig. 
403. Fig. 404 shows the 1440 Small Angle 
Divider being used with an auto collimator 
for the exact determination of the in¬ 
cluded angle of an optical prism. A wide 
range of applications is possible when the 
1440 Small Angle Divider is mounted on a 
Universal Measuring Machine (see pages 
315-316). 


FIG. 400 —For laboratory use, the IffO 
Small Angle Divider is furnished with 
granite plate, mahogany cab inet and 
protective plexiglass cover. 
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fig. %oi~-Thc HiO Small Angle Divider 
is mounted on the granite plate for the 
purpose of calibrating a photo-electric 
autoeollimaior over its full range. 


fig. 401 — T he il,h0 Small Angle Divider 
together with the standard granite plate in 
order to calibrate a precision rotary table. 


fig. 403 —A 13-sided polygon is calibrated 
through use of the mo Small Angle 
Divider. ■ 







FIG, 404 —Optical prisms of any angle arc 
ideally inspected with the IfW Small 
Angle Divider, 




PRECISION INDEX CENTER 

A circle-dividing instrument of a some¬ 
what different configuration is shown in 
Fig. 405. The Precision Index Center 
employs a horizontal roller bearing spindle 
of exceptional accuracy. Included in the 
design is a hardened, ground and lapped 
tool steel tailstock, and an accurately 
ground base plate to allow changing 
center-distance. 

This device is designed for fast, accurate 
indexing by means of a lapped carbide pin 
fitting into a lapped carbide hole in the 
back plate. Indexing increments are de¬ 
termined by master index plates with 
accurately jig-ground holes. An index center 
plate is selected with the desired number of 
holes in a circle (such as 24) and mounted 
concentrically with the spindle. 

While the number of indexed positions 
is limited by the selection of the index 
center plate, the emphasis in this design 
is on fast, repetitive, accurate indexing. 

The low profile of this device and the 
accurate centers in both head stock and 
tailstock make it especially suitable for the 
generation or inspection of angles of arbor- 
mounted parts. 

The Precision Index Center is shown 
in use in “Universal Measuring Machine 
Techniques & Applications,” page 313. 


fig. 405 — ' l ' he Precision Index Center is 
used especially for indexing parts Juicing 
centers or those that are best inspected 
while arbor-mounted, Its design enables 
very rapid, accurate indexing. 
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Roundness is required in many mechan¬ 
ical parts, as is trueness of rotation in 
spindles and many mechanical assem¬ 
blies. The authority for roundness usu¬ 
ally traces to a precision measuring 
spindle. 


fig. 406 —The Universal Measuring 
Machine Spindle, an essential element in 
the location process, is also employed in the 
inspection of roundness and form of 
circular parts. 


















1. Why Consider 
Roundness? 

The generation and measurement of the 
roundness of mechanical parts is the fourth 
art in the foundations of mechanical 
accuracy. The authority for roundness is a 
precision spindle, such as that of the Uni¬ 
versal Measuring Machine, Fig. 406. 

The importance of roundness has already 
been discussed in relation to circle-dividing 
equipment. It is difficult to choose the ex¬ 
act terminology. Some authorities suggest 
“trueness of form” or “cylindricity.” How¬ 
ever, since the intention here is to describe 
the generation and measurement of me¬ 
chanical parts wherein roundness is the 
essential form, it seems to be the appropri¬ 
ate term. For example, “trueness of form” 
might apply to many geometrical shapes 
unrelated to roundness; “cylindricity” 
does not encompass many categories 
within this art, such as the measurement 
of the roundness of tapered, circular bodies. 
Roundness has only recently been included 
in specifications of accuracy. Several years 
ago, when tolerances were generally ex¬ 
pressed no closer than a thousandth or 
even to several ten-thousandths of an inch 
[hundredths of a millimeter to thousandths 


of a millimeter], the actual roundness of 
holes, shafts and balls was seldom critical 
in a mechanical assembly. Machines and 
techniques in use generated shapes round 
enough to meet requirements. Under these 
conditions, there was little need for tool- 
makers, for example, to understand the 
nature of roundness. 

Tolerances of parts today, however, are 
commonly in the category of millionths of 
an inch [fractions of a /mi]. In retrospect, it 
would appear that space exploration stimu¬ 
lated the demand for higher precision. 
Now, increased accuracy is essential to 
sustaining a higher level of technology. 

The fact that “roundness” becomes pro¬ 
portionately more critical as tolerances be¬ 
come tighter is best proven by the simple 
example of fitting a shaft to a hole. Suppose 
we wish to determine the size of hole A to a 
tolerance of 50 millionths of an inch 
[0.0013 mm] for the purpose of inter¬ 
changeability, allowing little clearance 
with a perfect shaft B. Suppose, in addi¬ 
tion, that inspection reveals hole A is out- 
of-round by 50 millionths of an inch 
[0.0013 mm], but without a description of 
the nature of out-of-round. 

From studying two possible conditions, 
exaggerated in Fig. 407, it is seen that 
given a certain condition of roundness: 



50 millionths inch 


Axis of shaft alignment 
depends on type 
of roundness condition of hole 



50 millionths inch 


50 millionths inch — 
Axis of shaft alignment in hole 
depends on its roundness 


FIG. 407 . -(left) Locution, effective size and 

the fit between mating parts are all 
dependent on the ability to measure and 
define the condition of roundnc.ss. Hole “A" 
is out-of-round 50 millionths of an inch 


1 0.0013 mm\. From the two hypothetical 
conditions , it is apparent that more can be 
known of the functional relationship of 
hole “A” to mating shaft “B” if the 
roundness condition is clearly defined. 


fig. 4.08— firight) Shaft “B” in this case is 

out-of round 50 millionths of an inch 
[0.0018 mm}. How shaft “B” will function 
teeth mating perfect hole “/l” can be 

predicted, with more certainty if its 
roimdness condition is specified. 
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1. We are unable to specify the true or at 
least the effective size of hole A. 

2. The type of fit achieved, or clearance, 
is dependent on the roundness of hole 
A. 

3. Depending on the nature of the round¬ 
ness of hole A the axis of shaft B may 
be shifted. If this hole required refer¬ 
ence to some other hole in the part, its 
location would be in doubt. 

Where shaft B is not round and hole A is 
perfect, Fig. 408, it is obvious that similar 
limitations are imposed. 

It can also be noted by comparing Fig. 
407 to Fig. 408 that effective size and loca¬ 
tion of axes depends on whether the round 
shape is male or female. In most practical 
conditions, both mating parts will be im¬ 
perfect to varying degrees. 

Bearing in mind the limitations imposed 
by out-of-round conditions, some authori¬ 
ties suggest that the tolerance of roundness 
be five times closer than other related dimen¬ 
sional tolerances. If, for example, the shaft 
shown in Figs. 407 and 408 were a plug 
gage and a size tolerance of 50 millionths of 
an inch [0.0013 mm] were specified, then it 
would first have to be round within 10 mil¬ 
lionths of an inch [0.00025 mm]. If the hole 
shown in Figs. 407 and 408 were a ring 
gage specified to a tolerance of 25 mil¬ 
lionths of an inch [0.0006 mm], then round¬ 
ness of the ring gage must be within 5 mil¬ 
lionths of an inch [0.00025 mm]. Given 
this 5 to 1 ratio, it is clear that geometric 
conditions of roundness quickly limit other 
related tolerances, particularly size. 

Whether the suggested 20% ratio or 
some other proportion be used depends on 
the intended function of the part. What 
cannot be ignored is that there is a degree 
of uncertainty about other related toler¬ 
ances of a part, unless roundness is also 
maintained. Since the tolerance on round¬ 
ness should be much closer than that of 
the other dimensional tolerance which it 
effects, it is critical. 


fig. 410 — ( center ) The measurement of a 
dowel pin by micrometers shoivs a fairly 

fig. 409 —(top right ) Dowel pins, constant diameter at whatever point the 

mass-produced in a centerless-grinding measurement is made. Diametral methods, 

operation, are typically of constant diameter employing gaging points 180° apart, will 
but 3-lobed out-of-round. net reveal odd-numbered lobing. 


SOURCES OF 
OUT-OF-ROUNDNESS 

There are many causes of out-of-roundness 
when machining mechanical parts. Some 
common ones are: clamping distortion, 
spindle run-out, failure to “clean-up”, im¬ 
balance, heat and vibration. Their charac¬ 
teristic roundness shape varies greatly de¬ 
pending on the method of generation. 

NATURE OF 
OUT-OF-ROUNDNESS 

When the apprentice toolmaker carefully 
measures a dowel pin with a micrometer, 
he is puzzled when it will not slip into a 
hole measuring 0.0001 in. larger [0.0025 
mm], A higher quality gage pin which 
“mikes” the same size as the dowel pin, 
however, slips into the hole easily. Herein 
lies a lesson with regard to the nature of 
roundness. Lack of fit is due to the differ¬ 
ent methods of generation of the gage pin 
and the dowel pin. The gage pin is ground 
and lapped round. Dowel pins, mass-pro¬ 
duced, are only centerless-ground, for 
economy of manufacture. The shape of the 
latter is typically triangular, sometimes 
described as three-lobed, Fig. 409. 




bowel pin with 3-lobed out of-round characteristic 



FIG. 411 —Demonstration pieces prepared 
by the Van Keuren Company represent 
cylindrical pieces of constant diameter, 
each having a different number of 
odd-lobing. All the samples will stay in 
contact with the ruler when rolled 
through a full 360°. 
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FIG, 412 — Out-of'-roundness of ports 
having even-numbered, symmetrically 
arranged lobes, as with this 6-lobed 
cylindrical part, can be revealed 
by diametral methods. 



6-lobed, cylindrical part measured by 
diametral method. (Error discovered) 



Elliptical part I I 

1 Alignment of centers 

2 Surface condition of center holes 

3 Angle of centers 

4 Run-out of piece 

5 Roundness of centers 



Cylindrical part Centers 


fig. m—(center, left) Oul-of-roundness 
of an elliptical part becomes apparent by 
measuring across one diameter, turning 
the piece 180° and measuring across 
the second diameter. 

FIG. 415 — [center, right) A circumferential 
confining gage when used to measure 


fig. <113 — Out-of-roundness of -parts 
having an odd-number of lobes, as with 
this 7-lohed cylindrical part, is not 
revealed by diametral methods. 



7-lobed cylindrical part measured by 
diametral method. (Error not discovered) 

Circumferential confining gage 



Strangely enough, the centerless grind¬ 
ing operation produces dowels of a fairly 
constant diameter. No matter at which 
point one measures, the same diameter is 
always recorded, Fig. 410. 

Other shapes are shown in exaggerated 
form in demonstration pieces prepared by 
the Van Keuren Company, Fig. 411. All 
the samples will stay in contact with the 
ruler as they are rolled through a full 360°. 

If one tried to determine the size of a pin 
which was oval in section (or two-point 
out-of-round), he would discover a dis¬ 
crepancy by rotating the pin 90° and 
measuring once again. 

The point here is basic: 

1. All even-numbered (2, 4, 6, 8 etc.) sym¬ 
metrically arranged lobes in a part can 
be discovered by a diametral check, as 
in the six-lobed part of Fig. 412. 

2. All odd-numbered symmetrically ar¬ 
ranged lobes in a part will not be 
found by a diametral measurement, 
Fig. 413. 

It may be further concluded, then, that 
any diametral check will not necessarily dis¬ 
close effective size or roundness. This con¬ 
clusion applies to a cylindrical shaft, a ball, 
a hole, or to any other radial or spherical 
shape. 

All tools and instruments, regardless of 
magnification, which measure size by 
diametral methods (points 180° apart), are 
unreliable monitors in determining round¬ 
ness. 

2. Measuring Roundness 

Since diametral methods are inadequate, 
how then can roundness be measured? An 
American Machinist article* contained a 
symposium of opinions of gaging authori¬ 
ties on the subject of roundness. This 
article provides a basis on which to make 

*“And Must be Round Within . . American 
Machinist, December 1, 1958. 


roundness requires a separate highly accurate 
master for each size part to be measured. 
Other related geometrical characteristics 
may not be shown. 

FIG. Aid--- (bottom, left) Some parts may be 
inspected for roundness while mounted on 
coders. As here, there are many potential 


sources of inaccuracy which may lead to 
misleading results. 

FIG. 417 —( bottom , right) A 60° V-block is 
best for inspecting the out-of-roundness 
of 3-lobed cylindrical parts, since error 
is exaggerated. 








observations, since practically all methods 
used for gaging roundness were described. 
The article is also interesting historically, 
since it reflects a general awakening to the 
need for understanding roundness. 

The six methods of determining round¬ 
ness discussed in the article are: 

1. Diametral. 

2. Circumferential confining gage —a shaft 
is confined in a ring gage and rotated 
against a set indicator probe. 

3. Rotating on centers. 

4. V-block —piece rotated against a set 
probe: 

a. of fixed angle; 

b. of adjustable angle. 

5. Three-point probe (120° spacing). 

6. Accurate spindle. 

a. part fixed, exterior spindle with 
probe rotates; 

b. probe fixed, part rotates with 
spindle. 

Each of the six methods can be analyzed 
as to their effectiveness in measuring 
roundness: 

1. Diametral, Fig. 414 

It has been shown that the diametral 
method of measuring roundness is only 
suitable when the specimen is elliptical or 
has an even number of lobes. 

2. Circumferential Confining Gage, 
Fig. 415 

While this method may be useful for in¬ 
spection of roundness in production* it 
demands a separate, highly accurate 
master for each size to be inspected. The 
clearance between part and gage is critical 
to reliability. 

This technique does not allow for the 
measurement of other related geometric 
characteristics, such as concentricity, flat- 

*Canfield, a contributing author of “And Must 
be Round Within . . notes an accuracy “within 
10% of the 10% tolerance permitted by quality re¬ 
view,” p. 111. 


FIG. 418— (top) A 108° V-block is 
recommended to measure the roundness of 
5-lobed cylindrical parts. 


FIG. 410— (center) A 128° 3V V-block is 
recommended, to measure parts which are 
7-lobed. 


ness of shoulders, etc. The values obtained 
are still dependent on the shape of the 
specimen. 

3. Rotating on Centers, Fig. 416 

Certain parts, such as shafts or the OD’s of 
arbor-mounted parts, may be inspected for 
roundness while rotated on centers. Just as 
with the circumferential confining gage, 
this is far from being a universal method. 
Reliability is dependent on many factors, 
such as the angles, alignment, roundness 
and surface condition of the centers and 
center holes. Out-of-straightness of the 
part will cause a doubling runout effect and 
appear to be a roundness error. 

Any or all of these factors may combine 
in a given inspection, creating a high degree 
of uncertainty as to the exact nature of the 
error. 

4. V-block 

a. Fixed angle. Appleby and Worthen* 
recommend the use of a 60° V-block 
for discovering three-point out-of- 
roundness. Calculations would show 
that five-lobed parts are best meas¬ 
ured by a 108° V-block and seven- 
lobed parts by a 128° 34' V-block. See 
Figs. 417, 418 and 419. 

b . Adjustable V-block, Fig. 420. C. A. 
Whitney** observes that, considering 
the variety of machines and methods 
for generation of parts, often the num¬ 
ber of lobes is not known. To rectify 
this situation, he suggests a V-block 
which can be adjusted to the correct 
angle to show out-of-roundness after 
the number of lobes of the piece has 
been determined. 

Limitations of V-block method 

Use of a V-block of any angle is limited in 

the determination of roundness of parts, 

*Idem. 

**Idem . 




Part with unknown number of lobes 



fig. 420 — {bottom) When the number of 
lobes is unknown, a cylindrical part may be 
inspected, by a V-block whose included 
angle is adjust able. 
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FIG. 421 —Although a 60° angle V-block 
'is often thought most, perfect for revealing an 
out-of-round condition, results obtained 
are highly uncertain. Depending on the 


number of lobes in the part, that which 
actually may be shown is: no error (left), 
the error exaggerated (middle) or partial 
error (right). 


5-lobed part 3-lobed part Random-lobed part 



since it is based on the assumption that the 
number of lobes is known and uniformly 
arranged. 

The 60° V-block is often suggested as the 
best compromise to discover most types of 
out-of-roundness. However, depending on 
the number of lobes of the part being in¬ 
spected, it may show, Fig. 421: 

a. N o error (5, 7 lobed). 

b. The error magnified (3-lobed). 

c. Partial error (randomly spaced lobes). 

Seldom will the 60° V-block show true 

error. Similar uncertainty results when us¬ 
ing V-blocks of other included angle. The 
90° V-block, for instance, will hide a seven- 
lobed characteristic. 



5. Three-point Probe 
The three-point probe with 120° spacing, 
Fig. 422, is very useful for determining 
effective size where there is doubt as to the 
geometry of the part. The more common 
examples of a three-point probe are a three- 
jaw inside micrometer, or three-jet spindle 
for gaging lobing of a hole, or three-jet ring 
gage for gaging a shaft. When used for in¬ 
specting roundness, such gages do show 
three-lobing; however, having gaging points 
120° apart, they perform like a 60° V-block, 
with similar limitations. 

Production Methods vs. 

Absolute Methods 
of Measuring Roundness 

A distinction between production versus 
absolute methods of measuring out-of- 
roundness should be made. All the previous 
methods are suitable for production, and 
fairly inexpensive. The real danger in their 
use, however, is that those responsible for 
quality control may accept these methods 
as comprehensive checks. As previously 
observed, if a certain geometric condition 
of the part varies, then the roundness con¬ 
dition may be hidden or misinterpreted. 

None of these methods is satisfactory 
for measuring roundness absolutely. It is 


FIG. 422 — 3-point probes (points 120° 

apart) have similar limitations in 
measuring the roundness of holes as do 
V-blocks for measuring the roundness 

of cylinders. 
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FIG. 423 —The Spin Table is one example 
of the highly precise spindles which have 
become necessary to generate and to inspect 
round shapes. The Spin Table is a 


rotating table-type of spindle, useful for 
generating cylindrical, conical and 
spherical shapes when mounted on the 
jig grinder. 



therefore understandable that a demand 
arose for some universal means to provide 
a definitive value or calibration of round¬ 
ness, beyond the mere discovery of out-of- 
roundness conditions. 

6. Accurate Spindle 

It is now recognized that the only absolute 
method of calibrating roundness is with an 
accurate spindle.* There are innumerable 
types of spindle designs available today for 
the generation and inspection of round 
shapes. Various designs in use include: ball¬ 
bearings, roller bearings, plain ball spin¬ 
dles, plain bearings (journal type), metered 
oil flow and fluid-bearing (gas-bearing and 
oil-bearing). These spindles are available in 
a wide range of tolerances, down to a few 
millionths of an inch [fraction of a Mm]. 
Moreover, they vary widely in capability 
so as to meet particular needs. 

Spin Table 

An example of the “accurate spindle” is the 
“Spin Table,” Fig. 423. While the Spin 
Table may perform inspection operations, 
it has been designed for the generation of 
essentially round shapes. It is most versa¬ 
tile when used with the jig grinder for 
grinding highly accurate holes, outside 
diameters and shoulders. When the Spin 
Table is mounted on the Micro-Sine Table 
(see pages 213-215), tapers and conical 
shapes are ground round and to an exact 
included angle. If the Spin Table is placed 
on the jig grinder so that its spindle is hori¬ 
zontal, either by mounting it on an angle 
iron or on the Sine Table, spherical or 
aspherical shapes may be ground. 


*A recent development is a computerized device 
which enables roundness to be inspected on the ma¬ 
chine on which the part is generated, regardless of 
the accuracy of rotation. While useful for produc¬ 
tion, lesser accuracies are quoted with this method, 
and there are apparently certain lobing character¬ 
istics which are still hidden. 


The conventional design for this type of 
spindle includes the use of balls. For maxi¬ 
mum rigidity, and for minimizing the 
danger of brinelling, the Spin Table in¬ 
stead uses precision rollers under stiff 
radial and axial preload. 

Trueness of rotation of the Spin Table is 
held to 5 millionths of an inch [0.000127 
mm] Total Indicator Reading (TIR) and 
20 millionths of an inch [0.0005 mm] maxi¬ 
mum axial deviation, including the effects 
of camming, flatness and parallelism. 

Of particular interest to metrologists are 
those spindles designed strictly for the pur¬ 
pose of calibrating roundness. In rotation, 
these accurate spindles describe a perfect 
circle to which the part may be compared 
at all its radial ordinates. 

Out-of-roundness is shown by the move¬ 
ment of high-magnification comparators. 
It often suffices to merely observe and note 
indicator readings. However, the memory 


can hardly be trusted to visualize the 
geometry of a part as the spindle rotates, 
especially if its axis and that of the spindle 
do not coincide. Consequently, these spin¬ 
dles are usually equipped with means of re¬ 
cording the results. Sometimes linear re¬ 
corders are used but polar recorders pro¬ 
vide a better mental picture of the part, 
and perhaps for this reason have gained 
widest acceptance. 

The polar recorder is an auxiliary device 
having several rotational speeds, syn¬ 
chronous to those of the measuring spindle. 
Deviations from true roundness registering 
on the indicator are traced on circular 
graph paper at magnifications up to 20,000X 
to provide a permanent physical record. 

By overlaying a special transparent tem¬ 
plate where there are concentric circles 
representing calibrated values (such as 1 
line = 0.000010 in. [0.00025 mm]) to the 
graph, deviations from roundness may be 
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FIG, 424 —When visualizing the figundness 
condition of a part as represented in a 
polar tracing, a perspective of the magnitude 
of the error must be kepi in mind. 

The more the error is magnified, the more 
exaggerated the “star-shaped” picture. 
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Polar tracing of out-of round 
part, low magnification 


081 



Polar tracing of same out of-round 
part, high magnification 



easily understood and analyzed. This can 
be accomplished even without exact align¬ 
ment of the part with the spindle.* An 
alternative to the template method simply 
is to strike a circle on the chart with a draw¬ 
ing compass. The dimensional value put 
upon roundness, as well as the final center 
of the graphed circle, depends on the sys¬ 
tem adopted to assess roundness (see pages 
272-276). 

A perspective of the magnitude of the 
error must be maintained, since the graph, 
while truthful, is an enlarged, distorted 
representation of the error. The more 
scaled-up the representation, the more 
star-shaped it is, Fig. 424. 

3, Two Types of 
Measuring Spindles 

Roundness-measuring spindles are of two 
configurations: 

1. Overhead spindle, Fig. b25. The part is 
fixed while the overhead spindle, to 
which the comparator is attached, 
rotates separately from the part. 

2. Rotating table, Fig. i26. Integral to the 
spindle is a rotating table which car¬ 
ries the part and rotates it past a fixed 
comparator. 



MEASURING SPINDLES 

COMPARED 

Rotating Table 

It becomes apparent—compare Figs. 427 
and 428—that the rotating table spindle 
will determine more geometric characteris¬ 
tics without having to move the part. It 
measures such factors as: roundness, con- 

*R. E. Reason’s “Talyrond Handbook” examines 
the effect of non-centering and concludes that the 
shape of the graph is altered negligibly when the 
part is slightly off-center—up to 0.3 inch [7.62 mmj 
of the graph. Over this amount, distortion in¬ 
creases rapidly, becoming intolerable, when greater 
than 0.5 inch [12.70 mm] of the graph. 


fig, 12 - (center). When using the Overhead 
Spindle to measure soundness, the part is 
fixed, white the spindle carrying the 

compandor rotates about or within the part. 


fig, 420—( bottom) The rotating table-type 
of spindle carries the part to be measured 
for roiiHitnesS, and the measuring probe is 
fixed, usually on an indicator stand. 
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fig. 127 — (top) The geometric features 
of a pari that can be inspected with the 
rotating table-type of spindle are shown. 
This design allows the inspection of 
concentricity over a great range along 
the part axis. 


fig. 428— ( center) Shown are the 
geometric features of a part that can be 
inspected with the rotating table,-type of 
spindle. This design is especially suited 
to the inspection of roundness of large, 
heavy workpieces, or those where the 
part features to be inspected are, off-center. 


centricity, and camming (circular flatness). 
Squareness inspection requires sliding the 
indicator and stand, or an auxiliary hori¬ 
zontal movement. 

Unless there is a means for accurate ver¬ 
tical adjustment, it will not measure large 
tapers, Fig. 429. 

Since the indicator and its stand can be 
placed or adjusted conveniently to gage at 
any point, the allowable height of the 
workpiece is limited only to the extent that 
it becomes unwieldy. It is not restricted by 
the parameters of the setup. 

The rotating table must also carry the 
part. Therefore, the variable of workpiece 
load is introduced. When small parts such 
as bearings, small balls, miniature gyros, or 
ring gages are measured, no loss of accuracy 
is likely to occur. With heavy or eccentri¬ 
cally-mounted pieces, spindle accuracy may 
be reduced by deflection or by shifting 
loads, Fig. 430. Although at present air¬ 
bearing tables seem to have the greatest 
potential for sheer roundness accuracy, 
there is also evidence that this design may 
be the least able to absorb unsymmetrical 
or changing loads. 

A ball-loaded spindle, on the other hand, 
presents the greatest risk of brinelling crit¬ 
ical bearing elements when mounting heavy 
workpieces. 

Overhead Spindle 

It can be seen from Fig. 428 that the over¬ 
head spindle will determine roundness as 
well as camming. Since concentricity is in¬ 
spected only by extending the indicator 
from the spindle, the range of this check 
may be less than with the rotating table 
spindle. Flatness and squareness are in¬ 
spected only by physically sliding the 
workpiece past the indicator as on a sur¬ 
face plate; this latter technique may be 
employed providing one is confident that 
the datum surface used is flat. However, 
this is not a desirable method since it is 
best not to touch the part, once mounted, 




OVERHEAD SPINDLE 



Taper 



ktg. 429 — (bottom, left.) Neither the 
overhead spindle nor the 
rotating table spindle, by themselves 
will reveal taper in the specimen. 

fig. 430 —( bottom , right) Since the rotating 
table spindle must carry the part, spindle 


accuracy may be deteriorated by heavy, 
eccentrically mounted pieces; damage may 
occur when mounting the workpiece. 

No loss of accuracy should occur and the 
chance of damage is slight, when mounting 
heavy workpieces to be measured 
by the overhead spindle. 
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fig. 431 — The overhead spindle (left), 
will inspect camming. Unless an accurate 
horizontal movement is provided, the 
inspection of squareness, taper and 
flatness is accomplished only by sliding 
the tvorkpiece , which is usually a 


disadvantageous technique in practice. 
The rotating table (right), will inspect 
camming as with the overhead spindle. 
Squareness, taper and overall flatness are 
inspected only by sliding indicator and 
stand, which is poor workshop practice. 



until all inspection is completed to avoid 
thermal change. Moreover, the mounting 
surface will wear if this technique is prac¬ 
ticed often, and in moving the part, its 
orientation to the spindle axis is disturbed. 
Unlike the rotating table-type, there are 
limitations to the height of the workpiece 
which can be inspected with the overhead 
spindle. 

The overhead spindle does have advan¬ 
tages in use. Since the workpiece is station¬ 
ary and separate from the spindle, the spin¬ 
dle does not have the variable of workpiece 
load. There is no deterioration of spindle 
accuracy when measuring a part which is 
heavy or eccentrically mounted. Moreover, 
there is scarcely any danger of damaging 
the spindle during mounting of the piece. 


Measurement of Camming, 

Flatness and Squareness 

A commonly-held misconception is that 
both these types of spindles can inherently 
determine squareness and flatness. Both 
can measure circular flatness (variations in 
an axial direction in concentric circles from 
center), but without a controlled measuring 
movement in a horizontal direction, certain 
geometric errors may not be shown. For 
example, in Fig. 431, the shoulder of the 
piece is uniformly tapered (not square to 
the shaft) in the amount of 0.0001 inch 
[0.0025 mm]. The vertical spindle (left) 
may make circular tracings at selected in¬ 
crements along this shoulder. However, 
since the indicator adjustments are not re¬ 
lated or oriented, only camming, not 
squareness or flatness is shown unless the 
piece is manually moved. 

The rotating table has a slight advan¬ 
tage (right) in that it can show squareness, 
by sliding the indicator and stand, Fig. 
431. This technique also is not good work¬ 
shop practice. 

In Figs. 446 through 450, a study is made 
of other shapes that might be inadequately 
inspected with roundness-measuring spin- 
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fig. 482 —The locating process is dependent 
on the use of an accurate s/phulle. This 
spindle does not have a constant axis of 
rotation , shifting from C > to C 2 on every 
other revolution. The uncertainty of 
center-distance of hole “A" to hole “B” 
is attributable to spindle error. 


dies that do not include accurate horizon¬ 
tal movements. 

It is interesting that although very so¬ 
phisticated roundness-measuring instru¬ 
ments had been available for many years, 
only the Universal Measuring Machine 
could truly measure taper, flatness and 
squareness. Not until the Microtecnic Ex¬ 
hibition in Zurich, Switzerland, 1969, did 
the author see the introduction of con¬ 
trolled vertical and horizontal movement 
by manufacturers of these instruments. 

Effect of Temperature 

The rotating table is sometimes claimed to 
be inherently immune to the effects of tem¬ 
perature because the axis of the part and 
that of the spindle are mutually fixed. There 
may be truth to this claim where a time-lag 
exists between start and finish of a check. 
However, most roundness inspections re¬ 
quire only a few minutes. The effect of tem¬ 
perature variation on the column of the 
overhead spindle should not be substantial¬ 
ly more serious than the effect of tempera¬ 
ture-variation on the indicator stand used 
in conjunction with the rotating table. 



4. Moore Universal 
Measuring Machine 
Spindle 

NEED FOR AN ACCURATE 
SPINDLE 


It is absolutely essential that any measur¬ 
ing machine employed for accurate hole 
location be fitted with a spindle that rotates 
true. The importance of this requirement, 
often overlooked, is demonstrated in Figs. 
432 and 433. In Fig. 432, the spindle does 
not have a constant center of rotation, 
shifting from Cl to C2 on every other revo¬ 
lution. In Fig. 433, the spindle does not 
rotate true. Both examples reveal that 
when aligning the axis of the spindle with 



FIG. 433 —This spindle does not rotate true, 
causing uncertainty of actual center-distance 
of hole “A" to hole 'TV'. 
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fig, 43-1 —Accuracy of lhe Universal 
Measuring Machine spindle is verified 
by comparison to a master glass 
hemisphere, the latter round to 1 millionth 
of an inch [0.000025 mm]. The hemisphere 


itself is proved ff an identical tracing 
results when it is rotated and compared 
once again to the spindle. The transparent 
template in the foreground is useful in 
analyzing polar tracings. 



the axes of holes A and B for the purpose of 
measuring center distance, the error in the 
spindle makes it unlikely that the holes 
have been truly “pickedup.” Thecenter dis¬ 
tance measurement will most certainly be in 
error. Similarly, if the holes A and R are out- 
of-round, a true spindle will most accurately 
determine the correct centerline of the hole. 

Considering that the Universal Measur¬ 
ing Machine measures hole location to less 
than 35 millionths of an inch [0.9 pm], its 
spindle must be accurate to a few millionths 
of an inch [fraction of a pm] if the 5 to 1 
ratio of roundness to length is to be main¬ 
tained. Accurate spindles are inspected by 
a master round hole, or ball, such as the 
glass hemisphere of Fig. 434. A polar re¬ 
cording is made as the spindle is compared 
to the master. Accuracy of the ball is self- 
proved by turning it 90° and making a sec¬ 
ond polar chart. If the ball is not perfect, 
its error is shown displaced 90° on the sec¬ 
ond polar trace. 



MEASURING MACHINE 
SPINDLE HOUSING DESIGN, 
Fig. 435 

Spindle and Quill 

The two main elements of the Measuring 
Machine spindle are the spindle shaft and 
the quill. The tolerance of these parts as to 
size, straightness, and roundness is to a few 
millionths of an inch [fraction of a pm). 
Although the housing bore, the spindle 
shaft, and quill ID and OD are all ground 
accurately, especially as to straightness, 
the final accuracy can only be achieved by 
hand lapping, Fig. 436. Lapping produces 
an inherent accuracy that so far seems to 
be unattainable by alternative methods of 
machine generation. The reason for the 
superiority of the lapping process is that 
an averaging effect takes place. The ran¬ 
dom motion of the lapping produces sur¬ 
faces rounder and straighter than either 
the lap or the part is to begin with, Fig. 437. 


fig. 435 —The Spindle Housing is of such 
a design that when used in conjunction 
with other capabilities of the Measuring 
Machine, virtually any characteristic 
of size, form, rouniness or location may be 
determined to a high degree of accuracy. 
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fig, 430 - Final accuracy of spindle 
housing dements is achieved by 
hand-lopping. Spindle housing is mounted 
vertically while Us bore , consisting of two 
hardened and ground bushings, is lapped 
round, straight and to size. 


Of course, accuracy is only attained if the 
results of the lapping can be precisely de¬ 
termined at each stage. In Fig. 438, the 
spindle shaft is being measured for size 
and uniformity over its full length, follow¬ 
ing a lapping operation. In Fig. 439, the ID 
of the quill is inspected by an air gage in 
reference to a master ring gage. 

Between the spindle shaft and the quill 
ID are four hundred, y inch [3 mm] diam¬ 
eter balls held in a brass retainer, Fig. 440. 
To establish the correct preload, the size of 
the balls must be determined precisely. 
Uniformity of ball size is also a prerequisite 
to rotational accuracy of the spindle.* 

Balls of y inch [3 mm] diameter are used 
as a thrust between a collar on the spindle, 
and a shoulder on the quill to enable spindle 
rotation free of axial camming. 

The quill assembly slides vertically with¬ 
in the housing, guided by two lapped 
sleeves. Clearance is held to a maximum 
of 50 millionths of an inch [0.0013 mm], 
allowing only enough space for oil film 
thickness. 

The Yoke 

The spindle is elevated and lowered by 
means of a separately guided yoke assem¬ 
bly. The yoke has a double rack and inte¬ 
gral guide rods, and acts upon the spindle 
through a non-influencing pivot. 

Normally, instead of such a yoke design, 
a spindle is elevated and lowered by a pin¬ 
ion which works against a rack set into the 
quill itself. By eliminating the rack from the 
spindle, accuracy is enhanced three ways: 

1. It allows more perfect geometry of 
the quill to be established in lapping. 

2. The rack and pinion do not distort the 
spindle when it is elevated or lowered. 


*Many other designs were tried and discarded be¬ 
fore that described here was adopted. For example, 
a plain bearing oil-film spindle was found to develop 
an oil-wedge, causing a shift in spindle axis on every 
other revolution. The conclusion was reached that 
this is a characteristic of this type of design. 




Random motion during lapping 


fig. 43? —The random motions which 
arc imparted during the lapping process 
produce an averaging effect, resulting in 
shapes which arc superior in roundness, 
straightness, uniformity of size and finish. 
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FIG. 438 —Following a hipping operation, 
size and taper of the spindle shaft 
are inspected using an electronic indicator 
and stand. 



fig. 439 —The lapping operation is 
'performed in conjunction with accurate 
gages. Here the size of the inside diameter 
of the quill is determined by comparison 
to a master ring gage, using an air gage. 



fig. 440— Several hundred balls of identical 
size under an exact pre-load are used in 
the Universal Measuring Machine 
spindle. The balls are being loaded in a 
brass retainer. 
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fig. 441 — The slip ring allows an electrical 
signal to be passed from the gage head of 
the electronic indicator to the display meter 
mounted on the column of the machine. 
Electrical contacts must be f rictionless lest 
spindle rotation be influenced. 


3. The guide rods and guide-rod bush¬ 
ings, accurately made and located, 
prevent rotational movement of the 
quill during its vertical adjustment. 

The quill and yoke design described 
makes it possible to attain a vertical quill 
movement which is straight, aligned with 
the spindle housing travel, and square to 
the table top. 

Rotation of the Spindle 

The spindle is rotated by hand through a 
knurled knob at the top of the spindle for 
normal measuring applications, such as 
hole location or exploring the geometry of 
a hole. When a roundness measurement is 
made, the spindle can be trammed under 
power at the rates of 1, 2, or 4 revolu¬ 
tions per minute. A slip ring passes the 
signal from indicator gage head to ampli¬ 
fier, to allow for continuous rotation in one 
direction without the inconvenience of an 
electric cord wrapping itself around the 
spindle. The slip ring is positioned at the 
top of the spindle (the cord passes through 
the hollow spindle) to avoid using up 
spindle nose to table height. The design of 
the slip ring, Fig. 441, prevents the slightest 
influence on spindle rotation. 

Spindle Housing Travel 

The spindle housing is elevated or lowered 
by a large handwheel at the right of the 
column. Although primarily an adjustment 
to accommodate larger workpieces, the 
housing travel is accurate and straight (60 
millionths of an inch over 17 inches [0.0015 
mm over 432 mm]). It is often convenient 
to have an accurate vertical movement be¬ 
yond the range of vertical adjustment of 
the quill. For example, the inside diameter 
of the large stainless steel part in Fig. 442 
is measured top and bottom for concentric¬ 
ity and straightness. 

The construction features outlined assure 
that there is “good geometry” of the mov¬ 
able elements of the spindle housing in the 




FIG. 442— An accurate spindle housing 
travel at times proves convenient when the 
'piece to be measured requires a range of 
vertical adjustment beyond that of the quill 
alone, such as with this large stainless 
steel part. 




fig. 443 ---The. polar recorder, fastened to 
the right of the Measuring Machine, allows 
a permanent graphic record to be nmde of 
part features, such as nmndriess. 


The Universal Measuring Machine, thus 
equipped, can measure all required 
features of the ■part shown, such as hole 
sire, locution, roundness, straightness, 
squareness, and concentricity. 




process of coordinate measurement, con¬ 
tributing to overall accurate performance. 

Polar Recorder 

The polar recorder, Fig. 443, fastened to 
the right of the Measuring Machine, has 
rotational speeds synchronous to that of 
the Measuring Machine spindle. It pro¬ 
duces a permanent graphic record of round¬ 
ness, or other features of a part, such as 
concentricity, squareness and taper. 

5. Not Just Roundness, 
but Geometry 

Only recently has it become apparent that 
roundness specifications are a necessity. 
This may explain the present preoccupa¬ 
tion with roundness considerations, often 
to the exclusion of other equally important 
geometric features of a part. 

Out-of-roundness may not be the dominant 
error of the part. The fact that a hole is 
tapered by 20 millionths of an inch [0.0005 
mm] may be much more significant than 
being out-of-round by only 3 millionths of 
an inch [0.00008 mm]. As previously ob¬ 
served, however, it was not until the late 
1960’s that design features were included 
in roundness instruments to enable some of 
these additional geometric features to be 
measured. Although the Universal Mea¬ 
suring Machine was designed primarily to 
measure coordinate location, it was ten 
years ahead of roundness instruments in 
having the capability of measuring complete 
hole geometry. 

The significance of this point is best un¬ 
derstood by reference to Fig. 444, which 
contains a comparison of the inherent capa¬ 
bility of the rotating table, the overhead 
spindle and the Universal Measuring 
Machine to inspect various characteristics 
of form. An example of location relative to 
roundness is given in “Universal Mea¬ 
suring Machine Techniques and Applica¬ 
tions,” page 285. 
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Roundness 


Camming 


Characteristic of form 
to be inspected 


Overhead spindle 


Rotating table 


With no auxiliary vertical 
or horizontal movement 


Universal 

Measuring 

Machine 




Concentricity 


Alignment 


Straightness (directly) 





Barrel shape 


Hour glass 


Bell mouth 


Taper 


Squareness 


Flatness 


Relative size 


Absolute size 



Location 

(relative to roundness) 


Lean 

(relative to other part features) 



Only with auxiliary 
accurate vertical movement 


Only by sliding workpiece 
(overhead spindle) 
or indicator and stand 
(rotating table) 
or with 

auxiliary horizontal movement 




fig, 444 . -Three device s, the rotating 

table, the overhead spindle, and the 
Universal Measuring Machine, are 
compared as to their inherent ability to 
inspect various geometric forms 
relating to “-roundness 
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¥10. i:45 —Concentricity of two diameters is 
shown being measured on this polar tracing. 
The center of each diameter has been 
established by centering the glass template 
in turn over each tracing, then piercing the 
chart with a pin through a hole in the 
center of the template. 




MEASURING MACHINE 
FOR THE INSPECTION 
OF HOLE GEOMETRY 

The advantage of the Measuring Machine 
when used for the inspection of geometry 
of a hole, cylindrical shaft, or other round 
formsis that, in additionto ahighly accurate 
spindle rotation, all movements of the quill 
and housing are straight and parallel. Also, 
these movements are mutually square to 
the table top and to the plane of travel of 
the base slideways. 

Thus the Measuring Machine spindle, 
rotated by hand, shows roundness. More¬ 
over, when used to probe along the length 
of a hole (or outside diameter), it shows 
the “shape” of any side. In this straight¬ 
forward manner, taper, squareness, con¬ 
centricity and straightness are measured 
directly. This type of inspection is often¬ 
times much easier to interpret and more 
enlightening than a polar chart. 

Where a graphic record is desired, how¬ 
ever, these same geometric features may 
be shown on a circular chart using the 
Polar Recorder. For example, Fig. 445, 
concentricity of two diameters at different 
heights is determined by making a polar 
trace on the first diameter, then adjusting 
the quill vertically to make a trace of the 
other on the same chart. The transparent 
template is balanced out over the circular 
tracing representing each diameter, after 
which the graph paper is pierced through a 
pinhole in the center of the template. Lack 
of concentricity is then shown by the sepa¬ 
ration of the pinholes with an ordinary 
tool-maker’s scale. Such a measurement 
might appear crude until one remembers 
that the forms are represented at up to 
20,000x magnification. 

Very sensitive slideway movements of 
the Measuring Machine may be introduced 
at will where needed. For example, probing 
a shoulder for flatness, or centering a part 
under the spindle can be accomplished 


kig. cap - -There are errors of flatness on the 
faces of these models of lathe-turned pieces 
prepared by the Lawrence Radiation 


Laboratory of the University of California. 
The nature of the error has been deliberately 
obscured, as it -would appeur to the inspector. 
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fig. 447 —The most interesting sample 
prepared by the Lawrence Radiation 
Laboratory is one that has circular fullness 
over concentric circles of any diameter 
from, center, represented by the inked-in 
doited line. 


Overall, the piece is not flat. Accurate 
spindles, such as the Spin Table, 
show roundness of the outside diameter. 
Overall flatness is shown only by 
manually sliding the piece. 


without loss of orientation of spindle to part. 
An interesting example of the need for this 
latter capability can be demonstrated by 
models.* These models represent exagger¬ 
ated errors which might occur in lathe¬ 
turning.** 

Fig. 446 shows the four lathe-turned 
models, their errors deliberately obscured 
as it would be to an inspector. In Fig. 447, 
the most interesting of the samples is 
mounted on a Moore rotating table-type of 
spindle. An indicator probe is registered on 
the outside diameter, for measuring round¬ 
ness. It is apparent, however, that if polar 
tracings are made at only concentric circles 
from center on the top surface (one of which 
is represented by the dotted line), the piece 
would appear “fiat.” Yet the top surface is 
obviously not fiat. The top of the piece 
could be inspected for flatness by simple 
surface plate methods, such as the lathe- 
turned aluminum piece of Fig. 448. How¬ 
ever, when the piece is removed from the 
roundness-checker for inspection by totally 
different means, various part features, in¬ 
cluding the relation of the top face to the 
outside diameter are not correlated. 

Fig. 449 shows the four models mounted 
on the Universal Measuring Machine table 
where a variety of movements—rotational, 
vertical and horizontal—can be made. In 
addition, all such movements can be related. 
With this capability, virtually no geometric 
irregularity goes undiscovered. 

In Fig. 450, an actual part (outer race of 
rotary table spindle), requiring these capa¬ 
bilities, is mounted on the Universal Mea¬ 
suring Machine. The indicator probe is po¬ 
sitioned for a circular trace of the shoulder 
by spindle rotation; taper (squareness) and 
flatness of the shoulder are inspected by 


*Loaned by James Bryan, Chief Metrologist of 
the Lawrence Radiation Laboratory, University of 
California. 

**James Bryan, et al., “Spindle Accuracy,” 
American Machinist , December 4, 1967. 




fig. 448 —Flatness may be inspected 
separately from roundness by means of a 
surface plate, indicator and stand. When 
inspecting circular parts, however, as with 
this aluminum lathe-turned piece, other 
geometric features are not clearly related. 
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PIG. 44v —The models mounted on the. 
Measuring Machine Table represent 
exaggerated forms of typical errors which 
may occur in surfaces generated by turning. 
They dearly illustrate why related vertical 


and horizontal movements are required in 
addition to spindle rotation when measuring 
circular forms. 

Models courtesy of James Bryan of the University »f 
California, Lawrence Radiation Laboratory. 




traversing the X-axis across the shoulder 
width. Other irregularities, as in the dem¬ 
onstration models, are inspected by mov¬ 
ing both X and Y axes to probe anywhere 
around the shoulder. All other geometric 
features—roundness, straightness, flatness 
and concentricity—are inspected without 
moving the piece. Moreover, the size of the 
hole, top and bottom, and the size of the 
outside diameter are also inspected and 
can be correlated with overall hole geom¬ 
etry. 

Fig. 451 shows roundness and size of the 
outer race of an angular contact ball bear¬ 
ing being determined. In Fig. 452, the 
bearing race is mounted vertically on a 
form of V-block; a bent probe is employed. 
The piece is held in place with only light 
spring pressure, so that it may be rotated 
by hand to determine race geometry and 
size at any cross-section. To the right are a 
pick-up block and wrung-on gage block, 
necessary to relate the indicator probe off¬ 
set to the spindle when measuring size. 

Hole location, where required, may be 
correlated with size and roundness, all to 
virtually gage block accuracy. The tech¬ 
niques employed with the Measuring Ma¬ 
chine for establishing location and size as 
well as other uses are explained more fully 
in “Universal Measuring Machine Tech¬ 
niques and Applications,” pages 287-319. 

6. How to Define 
Roundness? 

RELATION TO SURFACE FINISH 

Unfortunately, as noted with the definition 
of “surface finish,” “roundness” has no 
universally accepted set of standards. In 
fact, surface finish itself becomes a consid¬ 
eration in gaging roundness. For example, 
it may be decided in the case of a lathe- 
turned piece, that the helical grooves which 
are typically produced in that operation 
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FIG. 450 —An actual part requiring 
geometric accuracies on all surfaces. 
Jn addition to its accurate spindle for 
measuring roundness, the Universal 
Measuring Machine has accurate 
horizontal and vertical movements to 
reveal any geometric irregularity. 


only relate to texture, not to actual out-of¬ 
round. It may also be decided that the 
probe tip used be of greater radius, so that 
the crest and root of the grooves are 
spanned, giving a truer picture of round¬ 
ness, Fig. 453. 

In analyzing a ball-bearing race for 
roundness, however, even finer considera¬ 
tions of surface texture are important 
criteria. It may be desirable, for example, 
to have a probe tip of smaller diameter, 
Fig. 454. 

In each of the two examples cited, round¬ 
ness values may be altered depending on 
the methods employed. 

DEFINITIONS OF 
ROUNDNESS COMPARED 

The definition of roundness is a highly com¬ 
plicated matter. It is not the purpose of 
this text to discuss the subject in great de¬ 
tail nor its relation to the allied subject of 
surface texture. However, it is instructive 
to briefly describe a few of the proposed 
definitions as well as the problems which 
may accompany any attempted definition. 

Assessments of roundness most often re¬ 
late to polar tracings produced by a round- 
ness-checker. Although it is generally agreed 
that radius, not diameter, should be the 
basis of roundness measurement, this one 
premise may still result in different inter¬ 
pretations. 

Some authorities, notably those in Ger¬ 
many, propose a “zonal” method where (1) 
the out-of-roundness value is the greatest 
deviation from the largest inscribing circle 
(a hole), Fig. 455; (2) out-of-roundness is 
the greatest deviation from the smallest 
circumscribing circle (a shaft), Fig. 456; or 
(3) out-of-roundness may be defined as the 
difference in radius of two concentric circles 
(having the same center), Fig. 457, one of 
which is the maximum which can just lie 
within the tracing and the other the mini- 



FiG. 451 —Roundness and size of a 

ball-bearing outer race are inspected on. the 
Universal Mensuring Machine. 
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fig. 452 — The ball-bearing race is mounted 
vertically and a cross-section of the 

ball-bearing race is measured for roundness 
and size, using a bent probe. Note the 


pick-up block and wrung-on block (right) 
used when measuring size (the gage block is 
required only to obtain a flat reference 
edge). 



i 


Large diameter probe 
used to span grooves 


-Helical grooves caused 
by turning operation 



Lathe turned piece 


Ball bearing race 


mum which can lie just without the tracing. 

Others propose that the difference in 
diameter between the largest inscribing and 
smallest circumscribing circle is the amount 
of out-of-roundness, Fig. 458. 

Some metrologists in the United States 
favor a “maximum radial deviation” defi¬ 
nition of roundness, essentially the same as 
the zonal method. R. E. Reason* points out 
that with the zonal method, it must be 
specified whether the chart is of a hole or 
a shaft, since each may have a different 
center, and thus different values may be 
obtained. The point is borne out by com¬ 
paring the tracings of Figs. 456, 457 and 
458. Reason shows that under certain cir¬ 
cumstances, the “radial” method also lacks 
a unique center. Perhaps because of these 
discrepancies, England has officially adopt¬ 
ed the more mathematically precise “mean 
circle” definition, or that of “. . . a circle 
such that the sum of the squares of equally 
spaced ordinates measured from it has a 
minimum value,” as originally suggested 
by R. E. Reason,** Fig. 459. The advan¬ 
tage of this system is that it does have a 
unique center. Its disadvantage is that it 
does not necessarily relate as well as radial 
methods in utilizing roundness measure¬ 
ments to determine effective size. The 
“mean circle” definition is not that helpful 
in determining, for example, where the 
axis of a shaft is most likely to be located 
in a hole. 

An approximation of the mean circle 
may be made by using transparent tem¬ 
plate or pencil compass. However, a true 
interpretation of values requires use of 
elaborate computer instrumentation, indi¬ 
cating that this definition cannot be as uni¬ 
versally applied, particularly for ordinary 
workshop requirements. This definition, for 


*R. E. Reason, “Some Basic Principles of Surface 
Metrology,” Engis Equipment Co., Chicago, Illinois, 
October 1, 1960. 

**Ibid. 


FIG. 453— A large-diameter probe may be 
preferred when measuring the roundness of 
lathe-turned pieces so that the helical 
grooves typically produced in feeding the 
tool (surface finish) are excluded as muck 
as possible from actual roundness, 
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FIG. 454— When measuring the roundness 
of a ball-bearing race, a small-diameter 

probe may be preferred so that more local 
variations (surface finish) are included 
in the roundness inspection. 





FIG. 455— (top left) The zonal method, 
favored, by many in continental Ev.ropi 
is sometimes recommended for defining 
roundness. When describing roundness 
of a hole, deviation should be related 
to the largest inscribing circle. 


fig. 456 . (top, right) When the zonal 

method is used to describe the roundness 
of an outside diameter, then, the deviation 
should be related to the smallest 
circumscribing circle. 


example, is not easily applied to visual as¬ 
sessments of roundness using only spindle 
and indicator. 

The preference of United States officials 
was given in “Military Standard Dimen¬ 
sioning”—MIL-STD 8 C, Oct. 1963. From 
1.3.16 Roundness Tolerance —“A zone de¬ 
fined by two concentric circles, within 
which an actual surface must lie.” 

This definition is qualified in 7.6.10.1: 

. . either the difference in diameters of 
two concentric circles . . . between which 
the surface so toleranced must lie (diam¬ 
etral method) or the width of the annular 
zone between such concentric circles (ra¬ 
dial methods).” 

From 7.3.5 relating to true position and 
roundness, “Either . . . may be specified. 
Such abbreviations or terms as DIA, TO¬ 
TAL, WIDE ZONE, TIR, ON DIA. must 
be used in symbols and notes to indicate 
that the total tolerance has been specified 
for true position, concentricity or round¬ 
ness.” Such symbols as R, EITHER SIDE, 
or ON R are used for “half tolerance” on 
roundness. Where either is not specified it 
will be interpreted as on “diameter.” 

The introduction of the term “diameter” 
in the above definition may be misleading, 
since without careful perusal, it could con¬ 
note an acceptance of diametral methods of 
checking roundness. This, of course, was 
not intended. 

Use of the concepts of “total zone,” or 
the difference in diameters of concentric 
circles, however, is convenient in that ap¬ 
proximately the same dimensional value 
can be used when describing the roundness 
of parts, whether inspected by diametral 
methods or by a precision spindle. Such is 
not the case when “radial deviation” is 
used since this is a “half-tolerance.” “MIL- 
STD-8,” then, appeared at least to favor 
the “zonal method,” while giving flexibility 
to the specifier as to whether it is a “diam¬ 
etral” or “radial zonal” deviation that is 
called out. 
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Two concentric circles (on radius) 


Approximate out-of-round value 



fig. 458 —( center , right) Roundness may be 
defined as the difference in diameter 
of two concentric circles, one, a 
maximum (within the tracing), the other 
a minimum (without the tracing). 


fig. 457— (center, left) Out-of-roundness 
may also be described in terms of the 
difference in radius of two concentric 
circles, one being the maximum 
which can be inscribed within the tracing, 
the other being the minimum which, can he 
circumscribed, about the tracing. 



o 

Circumscribing circle (shaft) 

.081. 



>--d 2 -- 

Two concentric circles (on diameter) 
D 2 — Dj = Out-of-round value 


fig. 459 —( bottom, left) 'The “mean circle” 
definition of round ness is officially 
sanctioned in England. The advantage 
of this system is its unique center. 

Its disadvantages are (hat it (1) does not 
by itself relate as clearly to function.; 

(2) is not as universally applied. 
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fig. -!-oO — The United States standard of 
roundness. Diagrammatic of United States 
Standard Drafting Practices, Dimensioning 
and Tolerancing for Engineering Drawings 
(USASI Y1D5D966). 

Reproduced with permission of the American Society of 
Mechanical Engineers, United Engineering Center, 

New York. 



os i 



Total zone 




40 millionths 
Radial 


20 millionths 
Mean circle 

All dimensions are in inches 


In a notice of January 1, 1968, “MIL- 
STD-8 C” was inactivated and superseded 
by USAS I Y14.5-1966. According to 
“U.S.A. Standard Drafting Practices, Di¬ 
mensional Tolerancing for Engineering 
Drawings, USASI Y14.5-1966” — 5.5.J 
Roundness Tolerance: “. . . a roundness 
tolerance specifies a tolerance zone bound¬ 
ed by two concentric circles in that plane 
within which the periphery must lie ... 

It is noteworthy that no mention is made 
in the USASI definition of “Diametral” or 
total deviations, but it has not been deter¬ 
mined if this method is excluded. The inter¬ 
pretation of that tolerance as applied to a 
cylinder is shown in Fig. 460. Size, it should 
be observed, is also made part of the speci¬ 
fication of roundness. Roundness of cones, 
cylinders and spheres is interpreted simi¬ 
larly. 

Neither the inactivated MIL-STD-8C 
nor the USASI definition of roundness at¬ 
tempt to resolve some of the finer aspects 
relating to the definition under varying 
conditions. At the time of writing, com¬ 
mittees of the U.S.A. Standards Institute 
are involved in composing an acceptable, 
more comprehensive definition of round¬ 
ness. Any forthcoming definition, however, 
would be expected to include specifications 
for defining the trueness of a spindle. 

The disturbing element in this lack of 
agreement on standards is that the round¬ 
ness of a part or trueness of a spindle can¬ 
not be judged or compared unless an ac¬ 
cepted definition of roundness is made. To 
illustrate, Fig. 461 shows three methods of 
specifying roundness, as applied to a polar 
tracing. It is most revealing that a part 
may be represented as approximately 80 
millionths of an inch, 40 millionths of an 
inch, or 20 millionths of an inch [0.002 mm, 
0.001 mm or 0.0005 mm] out-of-round- 
ness, depending on the method used for speci¬ 
fying roundness. The need for universal 
agreement on a definition of roundness is 
apparent. 


fig, 481 — Three different methods of 
specifying roundness are applied to the 
same out-of-round part. 

Depending on the definition, 
out-of-round may he approximately 


SO millionths of an inch, fO millionths of an 
inch, or 20 millionths of an inch [0.003 mm, 
0.001 mm, or 0.000S rnm\. 
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UNIVERSAL MEASURING 
MACHINE TECHNIQUES 
AND APPLICATIONS 






The No. 3 Universal Measuring Ma¬ 
chine represents the four mechanical 
arts brought to their highest level and 
embodied within a single facility. 


FIG. 462 —The No. 3 Universal Measuring 
Machine is shown with the HbO Small 
Angle Divider mounted on its table and 
the polar recorder fastened to the right. 
When the four mechanical arts — 
geometry, length, dividing the circle and 
roundness—are each brought to their 
highest level and used in combination in 
this single facility, a unique measuring 
capability results. 













moose 

























1. The Philosophy of 

the Universal Measuring 
Machine Use 

From the time that the principle of inter¬ 
changeable manufactured parts was first 
applied by Eli Whitney, the level attained 
in interchangeability has been dependent 
on existing measuring capability. The es¬ 
tablishment of size and location of holes, 
contours and surfaces has been a most im¬ 
portant part of the measurement scheme. 

At first, tolerances dealing with location 
were to hundredths of an inch [tenths of a 
millimeter], then to thousandths of an inch 
[hundredths of a millimeter]. During World 
War II, dimensions began to be expressed 
in one ten-thousandths of an inch [pm’s] 
tolerance, to be quickly followed in the 
post-war period by “split-tenths”. Now, 
specifications of millionths of an inch [frac¬ 
tions of a pm] are common. 

For many years, the jig borer and jig 
grinder could establish locations to mil¬ 
lionths of an inch [fractions of a pm], closer 
than it was possible to inspect these same 
locations by alternative means. It was nec¬ 
essary that inspection move into this area 
of locational accuracy. 

The Universal Measuring Machine, in¬ 
troduced for this purpose, was first thought 
of as only a laboratory instrument. Few 
firms seemed to require accuracy beyond 
that attainable with a jig borer. However, 
the tolerances which were once sought only 
in the metrology laboratory are now re¬ 
quired in the manufacture of parts in the 
workshop. Consequently, inspection on the 
Universal Measuring Machine has increas¬ 
ingly become an integral part of the manu¬ 
facturing process. It is this aspect of its 
application which will be stressed here. 

Relationship of the Four Mechanical Arts. 
The foundations of mechanical accuracy 
have been shown to be composed of four 
elements: geometry, length, dividing the 


circle and roundness. In previous sections, 
they have been analyzed as if they were 
distinct entities. Only occasionally has at¬ 
tention been drawn to the relationship of 
one mechanical art to another—such as the 
importance of roundness in dividing the 
circle, or the effect of geometry on length- 
displacement of a machine tool. In actual 
practice, such a fine distinction does not 
exist. 

This section, in contrast to those pre¬ 
ceding, will examine the practical applica¬ 
tion of these mechanical arts, brought to 
their highest level and used in combination 
in one facility. Once combined, a nearly 
infinite variety of measuring tasks can be 
performed to a higher level of accuracy 
than could be attained by applying any of 
the arts singly. 

Such a combination is shown in Fig. 462. 
The No. 3 Universal Measuring Machine 
embodies geometry and length. The 1440 
Small Angle Divider mounted on its table 
divides the circle. The machine spindle, used 
in conjunction with the polar recorder fas¬ 
tened to the right of the machine, measures 
roundness. 

The first part of this section will be de¬ 
voted largely to the techniques utilized in 
measuring a wide range of sizes, forms and 
configurations through use of this single 
facility and its ancillary equipment. The 
last part of the section will point out the 
dependency of advanced measurement (as 
in a ruling engine) on all of the mechanical 
arts, working together. 

Various geometric and linear accuracies 
which are incorporated into the Measuring 
Machine and which come into play in the 
measurement process (aside from those to 
do with “roundness,” already described) 
are cited in the Measuring Machine “Cer¬ 
tificate of Accuracy,” Fig. 463. 

The Universal Measuring Machine. Be¬ 
fore describing the types of applications to 
which the Measuring Machine is best 
suited, it is important to distinguish clearly 


between the reasons for its selection in 
preference to, say, a Coordinate Measuring 
Machine (CMM).* Both concepts of in¬ 
spection are useful, but in different ways. 
As stated in a previous section (page 164), 
the CMM is intended for rapid inspection 
of production parts. Since geometric fea¬ 
tures of the part are not inspected, and 
since the CMM is not always significantly 
more accurate than the machine which 
may have generated the part, usually it 
can only determine whether the parts in a 
production lot “pass”; if they do not pass, 
the CMM will not necessarily reveal why. 

However, where quality review demands 
that all of the parts in a lot be inspected, 
the CMM has made possible a great im¬ 
provement in quality control, and a great 
reduction in the time required for quantity 
inspection. 

The Universal Measuring Machine, in 
contrast, is too slow for quantity inspec¬ 
tion. Yet it measures many times more 
accurately than the machine or technique 
used to generate the piece, even in the case 
of very precise gages. For the calibration of 
hole location gages, it is virtually a primary 
standard. For example, CMMs, tape- 
controlled drills, and similar coordinate- 
system machines, are best calibrated by 
a hole-location gage, so that the complete 
system error is included. Many leading 
manufacturers of these machines now spec¬ 
ify that the Universal Measuring Machine 
be applied on a regular calendar basis to 
calibrate the hole-location gage, which in 
turn inspects the Coordinate Measuring 
Machine, Fig. 464. 

The Universal Measuring Machine is 
sometimes thought to be useful only as a 
“final referee,” for instance when there is a 
disagreement as to the accuracy of a part. 
While there may be some satisfaction and 
confidence imparted in knowing that there 

*“Measurement: quick vs close,” American Ma¬ 
chinist, July 3, 1967. 
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FIG, t (j,) - ~“Ceriificaie of Accuracy” of a 

typical No, 3 Universal Measuring 
Machine, indicating built-in accuracies. 


MACHINE. 


MOORE SPECIAL TOOL COMPANY, INC. 
CERTIFICATE OF ACCURACY 

FOR AVCO LYCOMING DIVISION 


Measuring Machine 


MOD£L__ 


SERIAL NO_ Mz22J- 


j +.000015 


WE CERTIFY THAT THE ACCURACY OF THIS 
MACHINE HAS BEEN DETERMINED BY REFER¬ 
ENCE TO MOORE MASTERS WHICH HAVE 

been calibrated from master standards 

CERTIFIED BY THE NATlONAt BUREAU OF 
STANDARDS. 


-.000080 -.000125 


Parallelism of Table 
Surface lo Travel 
— Full Travel — 


Parallelism of Table 
Edges to Travel 
- Full Travel - 



Squareness of Table Travel 
To Cross Slide Trove! 

— Full Travel — 


Table Sweep — 8” Dio. 
(See alher side) 


TRAVEL 

IN 

INCHES 


Squareness of Quill Housing 
Travel lo Table 
- Full Travel - 


- *060010 


; >£.000020 


I LEAD 
ERROR IN 
"MIUIOHTHS- 


Parallelism a f Quill Travel 
to Quill Housing Travel 
— Pull Travel — 



lead Error 

"Millionth! 


longitudinal Sere* Travel in Inches 



FiGt 464 — A hole-location gage used fe 
inspect a Coordinate Measuring 
Machine ( CMM ) is calibrated on a 
No. 5 Universal Measuring Machine. A 
precision bail is the most accurate means 
of determining Ike axes of the tapered holes, 
Co u r l esy of Brown and S h arp e Ma rs ufaciuri n g Co m pa ny. 
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fig. 405 -Locational errors in this work- 
piece are due to its lack of flatness. Such 
cm error is more readily found through 
use of the Measuring Machine, since 
clamping pressure can he greatly reduced 
compared to that needed, for machining. 


Workpiece clamped for machining 




Workpiece in “free” state 


Workpiece not clamped over supports 




exists a means by which disagreements as 
to accuracy can be resolved, the applica¬ 
tions of a Universal Measuring Machine 
have evolved far beyond such a narrow 
concept. 

BENEFIT TO THE 
MANUFACTURING PROCESS 

Any given inspection function is performed 
essentially for the purpose of improving 
the manufacturing process in some way. 
The objective is to make it possible for 
parts to be produced more accurately, at 
less cost, or to make them of higher or more 
consistent quality. 

The Universal Measuring Machine does 
not usually make “chips.” Its unique ca¬ 
pability is in being able to resolve the exact 
nature of the error of all part features, re¬ 
gardless of complexity. The most valuable 
function of the Universal Measuring Ma¬ 
chine is to disclose measurement informa¬ 
tion which in turn permits the improve¬ 
ment of the machines or methods by which 
the part is actually brought to finished 
form. 

The manner in which the Measuring 
Machine may aid the manufacturing proc¬ 
ess is best understood by referring to a few 
examples. 

Example No. 1—Workpiece Flatness 

Fig. 465. A series of holes was jig bored in 
a rectangular part which was clamped to 
accurate parallels. Without unclamping or 
disturbing the part in any way, hole loca¬ 
tion was inspected and found to be within 
tolerance. 

Subsequent inspection of the part on the 
Measuring Machine revealed locational er¬ 
rors as a result of holes being out-of- 
parallel. 

Closer inspection revealed that the piece 
was not flat, and had been bent flat only 
when clamped firmly for machining. Since 
the piece was mounted on the Measuring 


FIG. 466-- Inspection of workpiece using 
the Measuring Machine reveals that the 
holes are out-of-parallel, traceable to faulty 

clamping procedure in the machining 
operation. 
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FIG. 467 —Holes in this workpiece are to 
location, but of poor “geometry,” caused 
by a dull grinding wheel and excessive 
wheel pressure. Such errors are easily 
discovered through use of a Universal 
Measuring Machine. 


Machine solely for inspection, clamping 
pressure could safely be reduced. In its free 
state, the holes were not truly parallel. 

Solution: Improve flatness of the piece 
prior to machining holes to location. 

Example No. 2—Clamping Distortion 

Fig. 466. The workpiece was flat, yet the 
Measuring Machine found that the holes 
in the piece were not parallel. By retracing 
the operations performed on the piece, it 
was found that when the holes were bored 
in the piece, it was not clamped over its 
support, causing the holes to be out-of-par¬ 
allel once the clamps were released. 

Solution: The operator was made aware 
of the importance of clamping directly over 
the support parallels. 



Example No. 3—Grinding Techniques 

Fig. 467. Inspection with the Measuring 
Machine revealed that the holes in a hard¬ 
ened, jig-ground die were bellmouthed. The 
error was attributable to a dull grinding 
wheel and too much wheel pressure on the 
finish hole-grinding operation. 

Solution: The operator was instructed to 
dress the wheel more frequently and to 
“spark out” on the final grind. 

Example No. 4—Temperature 
Distortion 

Fig. 468. A polar tracing made on the Mea¬ 
suring Machine of a relatively thin-section 
internally ground part revealed a three- 
lobed out-of-round condition. The piece had 
been ground while clamped over three ac¬ 
curate support pads. 

At first, it was suspected that the error 
was due to distortion at the three clamping 
points. However, the condition was re¬ 
peated with subsequent pieces even with 
greatly reduced clamping pressure. 

It was finally discovered that uneven 
temperature was causing the error. The 
piece was cooler in the area surrounding 
the three support pads, where heat from 


Heat flow during grinding operation indicated by red arrows. 



fig. 468 —An out-of-round condition in 
this part, recorded with the Measuring 
Machine, was traced to uneven 
temperature distribution during grinding. 


\Wl 




PIG, 409 —This cast iron blank was jig 
bored, then measured by surface plate 
methods to test the accuracy of the 
machine on which it was bored. 


Machining methods were not performed 
carefully enough. Use of a Measuring 
Machine far surpasses surface plate 
inspection in determining hole location. 




Roundness check made 
from true coordinate 
locations, holes x and y 


One of two datum holes 


Without locating ability, roundness checks would have 
been made from x ; and y v rather than from x and y 


grinding was more quickly dissipated. 
When the piece returned to uniform tem¬ 
perature, local areas of the hole contracted 
by different amounts. 

Solution: The part was given full support 
on a lapped surface to help dissipate heat 
and to assure greater uniformity of tem¬ 
perature during grinding. 

Example No. 5—Inspection and 
Machining Techniques 
Fig. 469. For the purpose of verifying the 
accuracy of a jig borer under working con¬ 
ditions, a flat, cast iron part was mounted 
on its table, and holes were drilled and bored 
to a nominal size at random locations. 

This test piece was next inspected by 
surface plate methods, with the piece 
mounted on an angle iron. An indicator 
and stand, gage blocks and height gage 
were employed in the inspection. 

On the basis of this inspection, the in¬ 
spector concluded that the jig borer in 
question was capable of working only to 
a ±0.0002 inch [±0.005 mm] tolerance. 

Inspection of the piece by the Universal 
Measuring Machine disclosed errors in hole 
location, but in addition that the holes 
were not the same size, had been only 
rough-bored, and were not normal to the 
top of the workpiece. Moreover, the piece 
was not flat. 

From the nature of the errors revealed, 
it was a simple matter to reconstruct step- 
by-step the incorrect procedures by both 
the operator and the inspector. For ex¬ 
ample, when the test piece was measured 
in the unclamped position the holes were 
not to location and not parallel. However, 
when the test piece was clamped, duplicat¬ 
ing the conditions under which it was jig 
bored, the holes were parallel and the loca¬ 
tional error greatly reduced. 

Solution: The operator was informed 
that for such a critical purpose the ma¬ 
chining techniques should be performed 
with utmost care. All the rough machining 


FIG. 470— This drawing of an actual 
workpiece demonstrates the importance of 
relating roundness and location — a 
capability found only in a Universal 
Measuring Machine. 
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should be done first to remove stresses and 
get rid of heat. The clamps should be 
loosened next. The test piece is clamped 
once again very lightly, followed by a fin¬ 
ishing feather-cut, with all holes held to 
exact size. If the highest possible accuracy 
is sought, the piece should be stress- 
relieved after rough machining, then re¬ 
machined flat on its mounting surface, and 
lastly finish-bored to size with several light 
cuts. 

The inspector was advised that his 
method of inspection was not even as ac¬ 
curate as the means by which the test- 
piece was generated. Surface plate inspec¬ 
tion of hole location has many potential 
sources of error, such as: the degree of flat¬ 
ness of the surface plate or workpiece, or 
errors which occur when sliding the indica¬ 
tor stand. Size and parallelism of the holes 
also influence measured location. It is un¬ 
likely that the piece has been exactly 
oriented once turned 90° on the angle iron 
to measure the other axis. 

Such test-pieces are best calibrated on 
a Universal Measuring Machine, or fail¬ 
ing that, on an accurate jig borer or jig 
grinder. 

Example No. 6—Roundness in 
Relation to Location 

Fig. 470. Upon inspection with the Univer¬ 
sal Measuring Machine, two half-holes 
which “run into” one another in a special 
hardened and ground part were found to 
be out-of-round by 150 millionths of an 
inch [0.004 mm] when measured from their 
specified coordinate position. This condi¬ 
tion was attributable to the interrupted 
cut of the grinding wheel. 

Solution: A more open, softer grinding 
wheel was chosen; the wheel was dressed 
more often; and lighter cuts were taken on 
the finish grind. 

Note that the use of conventional round- 
ness-measuring instruments would have re¬ 
sulted in entirely misleading conclusions 


as to the amount or nature of the error 
since they lack locational capability. In 
fact, such instruments would have assumed 
the axes to be approximately at X x and 
Yi, obtained by balancing the polar trace, 
rather than at X and Y, the true axes. 

In this type of workpiece, roundness and 
location should be measured simultaneously. 

Example No. 7—Close Measurement 
and Mass Production 

Fig. 471. In order to increase die life, in¬ 
crease press speed and enhance quality, all 
the tools for the Model 24 Timex watch in¬ 
troduced in 1966 were specified to closer 
tolerances than previous tools. The front 
and back watch frame dies, particularly 
difficult progressive dies having many small 
punches and holes as well as embossing 
features, were specified to locational ac¬ 
curacies of 100 millionths of an inch [2.5 
nm]. 

The Universal Measuring Machine en¬ 
abled the Moore Tool Room Division to 
assure that such accuracy was being met. 
Occasionally, holes were even shifted 
slightly to conform to Measuring Machine 
inspection. In reference to this die, Mr. 
George Turley, Chief Tool Engineer of the 
U.S. Time Corporation states: “It is most 
important to have exact alignment between 
punch holder, stripper plate, and die block. 
Straightness of holes is particularly critical 
to prevent wear.”* Turley attests to the 
success of his insistence on locational ac¬ 
curacy (together with systematic die main¬ 
tenance), by pointing out that 5 years 
later, this die had produced 20 million 
watch frames (average 400,000 a month) 
and would produce at least 50 million more 
over its life. Obviously, the additional cost 
of the few hours on the Measuring Machine 
can be considered negligible when com¬ 
pared to total cost savings. Turley equates 

^Author’s conversation with George Turley, circa 
December, 1969. 


the importance of measurement to produc¬ 
tion in the true directness of the mechanic: 
“Without accuracy, you don’t automate.”* 

Example No. 8—Economics 
Achieved by Close Measurement 
A dramatic example of the benefit to high- 
precision manufacture from improved in¬ 
spection occurred in the early stages of the 
space program and coincidentally, shortly 
after the introduction of the Universal 
Measuring Machine. 

A precision parts manufacturer was faced 
with an unusual dilemma. The performance 
of many future space vehicles was depen¬ 
dent on the gyroscope he was to build, as 
part of the inertial guidance system. Ac¬ 
curacy requirements were not yet resolved. 
At this time, there was no available means of 
inspecting the part with any greater accu¬ 
racy than the jig borer used to produce it. 

*Idem. 



Greater concentricities add tremendously to die life. 


fig. 471- . The Universal Measuring 

Machine, by assuring greater concentricities 
between punch plate, die and stripper, may 
add greatly to the life of ultra-precise 
progressive dies. In this way, Ike Measuring 
Machine makes a substantial contribution to 
lowering the cost of production. 
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fig. 472 — Design and construct ion of the 
base, axes ivere described in “Geometry”; 
the lead screw and the derivation of its 
accuracy was traced in “Standards of 
Length”; and the design of the housing and 
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spindle elements were described in 
“Roundness." All these elements are 
combined and assembled with the proper 
controls into one operational Universal 
Mensuring Machine. 


The essential controls used in operating 
the Universal Measuring Machine, are 
listed. 









Given this uncertainty and the high 
stakes involved, quality control engineers 
of this firm felt they had no alternative but 
to specify much closer machining toler¬ 
ances. They were, of course, reluctant to 
do so, since this would result in a tremen¬ 
dous increase in cost of the project. Fred¬ 
erick C. Victory, then Chief Engineer of 
the Moore Special Tool Company, induced 
the firm’s project engineer to try what was 
at that time a new concept in inspection— 
theUniversal Measuring Machine. Through 
its use, the deviation of all features of the 
part from specified tolerance was deter¬ 
mined with a much higher resolution and 
certainty than previously attained. By 
keeping careful records of the inspection of 
all parts produced, as well as results of 
their respective performance tests, it was 
found that certain tolerances instead of 
being closer, could be safely increased, 
resulting in great savings.* 

In the author’s observation, and as these 
previous cases would suggest, the most suc¬ 
cessful installations of Universal Measur¬ 
ing Machines have been in facilities where 
a close working relationship existed be¬ 
tween the production and inspection de¬ 
partments. 

2. Universal Measuring 
Machine Techniques 

The measuring and control facilities which 
are used in operating the Universal Mea¬ 
suring Machine are called out in Fig. 472, 
included here as an aid to understanding 
the techniques to be discussed. Additional 
information is also found in “Maintenance 
and Operation Manual: Model No. 1J4 
and No. 3 Moore Universal Measuring 
Machine.”** 

*Frederick C. Victory, The Moore Universal Mea¬ 
suring Machine—A New Concept in Measurement and 
Inspection, Publication of Moore Special Tool Com¬ 
pany, circa 1959. 

**Publication of Moore Special Tool Company, 
1967. 


MOUNTING 

A description of the techniques of using the 
Measuring Machine most properly starts 
with the mounting of the workpiece. 

The piece to be measured must be flat 
and supported on accurate parallels. If its 
flatness is in doubt, it can be checked 
directly by registering the indicator against 
the surface under consideration, and exam¬ 
ining its accuracy while traversing the 
machine axes. 

Clamping pressure on the workpiece 
should always be directly over its supports, 
and only finger-tight. A set of special 
knurled clamping nuts, a sampling of which 
is shown in Fig. 473, is furnished with the 
machine to assure adherence to this prac¬ 
tice. 



fig. 473 —A special set of workpiece 
damps is furnished with the Universal 
Measuring Machine; a few such items are 
shown. An important feature of the set is 
the use of knurled, nuts to avoid distortion of 
the workpiece through damping-pressure. 
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BASICS OF MEASURING 
HOLE-LOCATION THROUGH 
THE USE OF COORDINATES 

The coordinate system provides a most 
perfect way to measure location, since 
through its use the configuration of vir¬ 
tually any shape in one plane can be ex¬ 
pressed in numerical terms. 

The simple, rectangular part, Fig. 474, 
may be used to demonstrate most of the 
basic procedures. The nominal dimensions 
of this part are, 4 inches in width by 8 
inches long [101.6 by 203.2 mm]. The four 
1-inch diameter holes are 5 inches apart on 
its length and 2 inches apart on its width 
[25.4 mm holes, 127 by 50.8 mm apart]. 

Example No. 1 

Measure the location of the four holes with 
respect to a datum given as the center of 
the part itself. First the dimensions are 
converted to coordinate locations, refer¬ 
encing from the centerline of the part, as in 
Fig. 475. The whole-inch figure [convenient 
metric number] is selected simply by ap¬ 
proximating by eye the centerline of the 
part, when mounted, with respect to the 
nearest whole inch [or convenient whole 
millimeter figure] on the reference scale of 
each axis (see also readout page 309). 

Procedure 

1. Register the indicator probe on the 
side of the piece, and traverse the 
long axis. Hold one end of the piece 
and tap its side at the opposite end 
to bring it into alignment with the 
axis, Fig. 476. Clamp the part and 
repeat this process. 

2. Move the long axis until the spindle 
by eye seems centered over the piece 
longitudinally. This is accomplished 
by extending the indicator linkage 
and noting if the indicator probe 
appears to be directly over each end 
when swung through a 180° arc. 











3. Using the lowest magnification, 1 
line = 0.0005 inch [1 line = 10 pm], 
register a “zero” on one end of the 
piece at the lowest reading of the arc. 

4. Without moving the long axis, note 
the lowest reading of the indicator 
against the other end when swung 
through an arc. 

5. Leave the indicator set against the 
latter end and move exactly half-way 
between this reading and zero on the 
meter. The fine-setting knob facili¬ 
tates moving in such fine increments. 
Note: If the indicator does not regis¬ 
ter “on the meter” when swung to 
the opposite end from which the zero 
had been set, move the long axis 
until the indicator is brought up to 
zero at the highest point of the arc. 
Add the first coordinate setting to 
the second coordinate setting and 
divide by 2. Move to this position. 
Backlash. Backlash has been inten¬ 
tionally built in at the factory and 
has no effect on accuracy, provided 
that all final approaches are in a 
direction indicated by the arrows on 
the lead screw dials. 

6. Repeat the previous process to 
center the spindle over the part in the 
direction of the short axis. 

7. Switch to a fine scale on the meter; 
one division = 10 millionths inch 
[1 division = 2/10th pm on metric 
machines] is a convenient magnifica¬ 
tion. Then, reset each axis as before. 

8. Clamp both axes of the machine. 

9. Set the rough reference scales to the 
nearest “even inch” (or nearest 
10 mm increment). Next, set the dial 
and vernier to “zero,” Fig. 477. 

10. Re-verify the setting. Use vernier 
fine adjustment for the final adjust¬ 
ment of the vernier. 

11. Traverse to the nominal X-Y coor¬ 
dinate for one hole. Adjust the indi¬ 
cator against one side of the hole to 






FIG. 479— The required dimension of the 
holes is their distances from the centerline 
of the part (top). Once the coordinate 
indues are determined, they are easily 
calculated (bottom). 


get a zero on the meter. Rotate the 
spindle 180° so that the probe regis¬ 
ters against the other side of the 
hole. The meter will read double the 
amount that the hole is off-location 
on that axis. Then, move half the 
amount with the lead screw as in 
Fig. 478 and set zero on the indicator 
once again. 

12. Repeat this process for the same hole 
on the other axis. 

13. Record coordinate settings of both 
axes. 

14. Repeat steps 11-13 to find the co¬ 
ordinates of each of the other three 
holes. 

15. The distance of the holes from the 
center of the piece on each axis is 
now evident simply by subtracting 
the coordinate values obtained, Fig. 
479. 

Example No. 2—Two Line-Up 
Holes as a Datum 

In Fig. 480, the datum is specified to be 
the holes themselves, the outside of the 
part not being functional. 

The best procedure is still to start with 
the previous steps (Example 1, Steps 1 
through 7). Machine operators will usually 
bore or grind the holes in an approximate 
relation to the edges of the piece, even if 
this is not specified. It is more convenient 
to tap the piece into rough alignment by 
using an edge rather than two holes. 

Once the edge is aligned, and the spindle 
centered, the holes themselves will almost 
always fall within the coarse range of the 
meter, which is ± 0.010 inch [± 0.2 mm in 
metric], unless drastically out-of-line with 
the edges of the part. 

Orient the spindle so that the indicator 
registers against the back side of one hole. 
Set a zero at the lowest point of the hole 
found when traversing the long axis. 
Traverse the long axis to the other end 
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fig, 4 8#.*~ Frequently, the edges of a part are 
■nonfunctional, requiring only hole location 
(top)- Two holes, (A & B), are chosen 
for a datum for line-up and for 
convenience (bottom); the nominal “zero" 


may be set at hole A. Once coordinate 
locations are determined, distances from 
datum h»le A can he calculated 
arithmetically. 


FIG. 4Sl —The same techniques used to 
determine hole location of the L-hole 
demonstration -piece are applied to 
determine locations of more complex parts 
such as this precision progressive die. 
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Distances from hole-to hole are determined in the 
following manner (for instance from A to B on the X axis): 
11.499980 -6.500000=4.999980 


All dimensions are in inches 
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When the rotary table is employed to 
■measure by polar coordinates as shown 
then it must he. accurate as well. 


fig. 4S2 —The rotary table may be used to 
facilitate line-up of datum surfaces or 
holes of workpieces for measurement by 
rectilinear coordinates; in such cases, 
rotary tuble accuracy is not essential. 


and drop the probe into the other hole. 
Note the meter reading at the low point 
of the hole, again found by moving the 
long axis back and forth. 

Leaving the indicator set at the low 
point, hold the other end of the piece and 
tap it on the side at the end adjacent to the 
indicator probe until the meter records the 
previous zero. 

Switch to a high magnification. A slight 
final tapping adjustment may have to be 
made by indicating the full circumferences 
of both holes if they are still not exactly in 
line, due to slight differences in size or out- 
of-roundness. 

If the datum holes are not the same 
nominal sizes, initial line-up is still facili¬ 
tated by using one side of the holes. The 
cross axis in this case is moved an amount 
equal to half the difference in their nominal 
diameters to register the indicator probe. 
Final line-up is again accomplished by in¬ 
dicating the full circumference of the holes, 
except that the offset of the indicator probe 
is physically adjusted in order to register 
on the meter. 

These same techniques may be employed 
in measuring the location of holes in any 
workpiece, such as. a large die section, Fig. 
481. 

The location of outside diameters, such 
as those seen in Fig. 481, is accomplished 
using the same principles of operation. 

Example No. 3—Rotary Table 
Techniques 

A. To Facilitate Line-Up 

Some workpieces are too awkward in shape 
to tap into alignment; such a part is shown 
in Fig. 482. Besides, they may have no 
convenient reference edges to obtain an 
initial coarse alignment, and occasionally, 
the specified datum holes may not be in 
line. In these cases, a rotary table is used 
to align reference holes or surfaces with the 
measuring axes, as illustrated in Fig. 483. 




fig. 483 —The advantage of the r»tary table 
for part alignment with the measuring 
machine axes is that all adjustments are 
monitored. In contrast, hand-tapping 
the part into alignment depends more on 
operator technique and “fed.” Both 
techniques are necessary at times. 













fig. 484— {top) Measurement of hole location 
by polar coordinates, still requires use of 
one machine axis to establish the center- 
distance at which radial values are to 
be determined. 


fig. 4S5 --(center) To gain more working 
clearance for the measurement of small 
diameter holes, the back of the indicator 
probe (opposite the measuring direction ) 
may be relieved. 




Thin piece (less than .045 inch) Bottom-relieved probe 



If the dimensions are specified from a 
center-hole in the part, initial line-up is 
greatly facilitated by using the reference 
hole in the center of the rotary table. The 
Measuring Machine spindle is aligned with 
the center hole of the rotary table prior 
to mounting of the workpiece. Once 
mounted, the workpiece is tapped to bring 
its center hole central to the machine 
spindle axis. The alignment of datum holes 
is now achieved simply by following pre¬ 
vious procedures, except that the piece is 
rotated rather than tapped. 

Accuracy of the rotary table in this ap¬ 
plication is not critical, since it is used 
only for orienting the workpiece, not for 
measuring. 

B. To Measure by Polar Coordinates 
If the rotary table is to measure angles, 
such as of the holes of the part shown in 
Fig. 482, its accuracy must be commen¬ 
surate with that which must be achieved on 
the part. Equipment is available to enable 
measurements to be performed in a range 
of accuracy from ± 12 seconds to ± 0.1 
second of arc. 

The Measuring Machine spindle is 
initially aligned with the center bushing of 
the rotary table. Then the piece is tapped 
central to the machine spindle, as in the 
previous example. However, the center 
hole of the rotary table is not absolutely 
concentric with the rotary table axis and 
may be in error by a few millionths of an 
inch [approximately 2 nm]. 

The center hole of the part must therefore 
always finally be adjusted concentric with the 
rotary table axis. Alignment of a hole with 
the rotary table axis is accomplished in 
the following manner. The probe is reg¬ 
istered against one side of the center hole 
in the part. The rotary table is rotated 
through a full 360° and the misalignment 
of the hole is recorded at the 90° incre¬ 
ments. Now, by rotating the machine 
spindle, the probe is oriented normal to 


fig. 486 —Cor the sake of convenience, 
small thin parts, such as die stampings, 
may be measured without clearance under 
the holes by relieving the bottom of the 
indicator probe. Care must be taken not to 
relieve the probe beyond its “equator.” 
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the 0° and 90° increments of the hole, 
and at each of these increments the piece 
is tapped to centralize the hole according 
to the deviations which had been re¬ 
corded. This process should be repeated 
until the hole is aligned sufficiently close 
for the accuracy sought. 

Once the center-hole of the part is 
aligned with the rotary table axis, the part 
is clamped. By moving the coordinate 
axes, the center hole of the part is located 
centrally with the machine spindle and a 
coordinate setting established. 

The next step is to make an excursion 
equal to the radial distance from center 
to the holes to be measured, Fig. 484. 
Using the rotary table for rotation, the 
holes are then indexed one by one until 
located centrally with the measuring ma¬ 
chine spindle. Angular spacings of the holes 
are read on the rotary table dial. 

Unlike using a dividing-head and surface 
plate inspection, the Universal Measuring 
Machine is able to determine the center- 
distance of the holes, as well as the angu¬ 
lar spacings. 

Following this measurement, the piece 
may be double-checked by converting 
polar to rectilinear coordinates and mea¬ 
suring purely by the machine axes.* 

Example No. 4—Measuring 
Coordinates of Small Holes 

A. With Relieved Probe-Tip 

Small holes may be measured through use 
of probe-tips as small as 0.010 inch in 
diameter [0.25 mm]. Additional working- 
clearance may be gained in small holes by 
back-relieving the probe, Fig. 485; how¬ 
ever, the radius of the probe should always 
be smaller in size than that of the hole. If 
the part to be measured is very thin, and 
it is inconvenient to have through-clear- 

*Rectilinear coordinates may be calculated or 
found by referring to the “Woodworth Tables,” 
Holes, Contours and Surfaces by Richard F. Moore 
and Frederick C. Victory, pp. 227-414. 


Flu. 48 ? —The location of a small hole 
may be determined by using a gage pin 
to represent, its axis. The advantage is that 
probe-tips of a diameter greater than that 
of the hole may be employed. 



ance under the part, the probe may be 
relieved on the bottom, as in Fig. 486. 

The most accurate measurements of 
small holes are obtained through use of an 
electronic indicator, so this additional 
preparation, if necessary, is worthwhile. 

B. With Gage Pin Inserted Into Hole 

A gage pin may be used to represent the 
axis of the hole, as in Fig. 487. One ad¬ 
vantage of this technique of measuring the 
location of small holes is that the probe-tip 
diameter may be larger than that of the 
hole to be measured. The probe should still 
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FIG. 488 — When using a gage pin to 
represent the axis of the hole, precautions 
must be taken to: (i) Check the fit of pin; 
( 2 ) Check Us straightness; (3) Indicate as 
near as possible to the hole. 



be of relatively small diameter, nonethe¬ 
less, to minimize the error which may occur 
from the probe-tip not being exactly 
normal to the gage pin. 

To be reliable, this method requires that 
the pin be straight, does not lean and fits 
the hole closely. Straightness and lean are 
checked by running the indicator probe 
along the length of the pin. Fit of the pin 
is checked by applying pressure, noting 
if the pin always returns to position after 
the deflecting force is removed, Fig. 488. 
It is good practice to measure as close to 
the hole as possible to minimize the intro¬ 
duction of erroneous readings from either 
condition. 


C. With Microscope 

The microscope is especially useful for 
measuring the location of small holes or 
shapes such as of the watch part shown in 
Fig. 489. A microscope is also necessary 
for measuring parts which might be dam¬ 
aged by physical contact. Some parts, for 
example, the micro-circuit component, Fig. 
490, do not have any reference surfaces 
against which an indicator probe-tip may 
register, and also require the use of a 
microscope. 

A rotary table is usually indispensable 
for all microscope work since the work- 
pieces would otherwise be difficult to 
align. However, the rotary table need not 
be extremely accurate if used solely for 
orienting the piece. 

The Standard Microscope 

The convenience, versatility and range of 
magnification of the electronic indicator 
cannot be embodied into a single micro¬ 
scope. For this reason, the standard 30- 
power microscope shown in use in Figs. 489 
and 490 is furnished as standard, and 30- 
power has been selected as the best com¬ 
promise for all-around use. At less than 
30X magnification, the field of vision would 
be greater but accuracy of resolution would 



fig. 489 —The measurement of location of 
small holes and surfaces, such as of the 
watch part shown, are sometimes more 
connmdenibf accomplished through use of 


the standard 30 X microscope. The rotary 
table is indispensable to orient the pari 
whether or not dimensions are specified 
by polar coordinates. 
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FIG. 490 —Inspection of the micro-circuit 
component, shown mounted on the 
optical stage, requires use of a microscope 
A collimated light source located below 
helps to dearly define the functional 

StlTj CICt'S . 








usually be inadequate. At more than 3OX 
magnification, repeatability would be en¬ 
hanced but the field of vision would be too 
narrow and focusing would be unduly 
critical. 

The standard microscope has a field of 
vision of approximately 0.240 inch [6 mm] 
diameter. 

The reticle aids in centering the axes of 
holes or in orienting other part features. It 
is made up of concentric circles which in¬ 
crease in value by 0.006 inch [0.1 mm in 
metric] diameter up to 0.036 inch [1.0 mm 
in metric] diameter of the reticle, and by 
0.012 inch [0.2 mm in metric] diameter for 
the remainder of the 0.240 inch [6 mm in 
metric] field of vision. 

Included with the standard microscope 
is a transparent stage for mounting of the 
part. A collimated light source is placed 
below the transparent stage so that when 
viewed through the microscope, part fea¬ 
tures are clearly defined. 

Other Optical Arrangements 

For the user with a specific type of work 
for which a nonstandard microscope seems 
in order, it is preferable to select a partic¬ 
ular adaptation through engineering con¬ 
sultation. 

For example, one user had the require¬ 
ment of fast measurement for production- 
type work as would be achieved with a 
CMM, but also the requirement to mea¬ 
sure gages to a high order of accuracy, as 
would be achieved with a Universal 
Measuring Machine. However, the work¬ 
load in each category was insufficient to 
justify the expense of a machine for either 
category of inspection alone. The solution 
was to fit a Model No. 13^ Universal 
Measuring Machine with a Stocker & Yale 
visual comparator-type readout at a lower 
magnification of 20 X, Fig. 491. This mag¬ 
nification was sufficient for the production- 
class of work. In this arrangement, the 
viewing screen can be quickly swung clear 












FIG, 493 —A 400X measuring microscope 
is used to inspect a pattern of holes of 
.0015 in. [O.OSS mm] diameter, produced 
on a iV*. 8 Measuring Machine adapted 
to small hole drilling. 


to return to normal electronic indicator 
use for gage-type inspection. Fig. 492 
shows the image of a hole projected on the 
viewing screen. Production-type inspection 
was performed particularly well with this 
design since quality control of this firm 
specified the use of “true position toleranc- 
ing.”* The technique employed is interest¬ 
ing. Rather than orienting the holes con¬ 
centric with the rings on the screen, and 
recording their actual location, the lead 
screw moves the axes to the specified loca¬ 
tion. Two concentric rings on the screen 
are selected (or drawn) as minimum and 
maximum diameter tolerance zones to 
determine immediately if the hole falls 
within true position tolerance. 

In Fig. 493 is shown a 400-power mea¬ 
suring microscope. It is used to inspect 
size, location and other characteristics of a 
pattern of holes of 0.0015 inch [0.038 mm] 
diameter, which have been produced by 
means of a Moore small-hole drilling ma¬ 
chine. 

Example No. 5—Measuring Radius 
Size (Edge-Finder-Method) 

In generating holes and contours the jig 
grinder operator must continually make 
use of a technique called “indicator 
measuring,” first described in Holes, Con¬ 
tours and Surfaces.** 

The procedure is to first give a coor¬ 
dinate location to each hole or contour as 
shown in the section of a motor lamination 
die, Fig. 494. This die section has a total of 
21 holes and male and female radii, as well 
as several straight sections. It is readily 
apparent that while the coordinate loca¬ 
tions may be determined in this manner, 



^‘Dimensioning and Tolerancing for Engineering 
Drawings,” Bulletin USASI Y14.5-66, American 
Association of Mechanical Engineers, New York, 
pp. 41-84. See also R. F. Utter et al., “Concepts 
of the True Position Dimensioning System,” Sandia 
Corporation, Albuquerque, New Mexico, 1965, pp. 
36-100. 

**Richard F. Moore and Frederick C. Victory, 
Holes, Contours and Surfaces, p. 140. 














fig . i<)4 — This blanking station of a 
progressive die has 21 Male and Jemal 
radii, most of which the jig grinder 
operator must size by “indicator 


measuring techniques, the same 
techniques are ■adaptable to the ||j live-real 
M c muring Machine to perform a variety 
of measuring tasks. 
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fig . 495 -The “edge-finder’’ gage may be 
used as on® method of lace ling an edge far 
indicator 'meamrina. 





fig, 496 —Demonstrated is how a located 
edge may he used as a reference to 
establish the size of a female radius. 


most of the recessed radii are not accessi¬ 
ble for measurement of size by conven¬ 
tional tools such as bore gages, microm¬ 
eters or gage pins. 

The lead screw has up till now been con¬ 
sidered only as an element to establish 
location. Since the 35 millionths of an 
inch tolerance over 18 inches [0.0009 mm 
over 460 mm in metric machines] is often 
superior to that which can be obtained 
commercially in most length standards, it 
is logical that it be applied to measuring 
size as well. 

The employment of the indicator mea¬ 
suring technique is an absolute necessity 
for the jig grinder operator. Once these 
same techniques are understood and ap¬ 
plied to the Universal Measuring Machine, 
not only can size be measured, but a wide 
range of other measuring applications is 
possible. 




MEASURING FEMALE RADII 


The Edge-Finder Method 
The measurement of size through use of 
the Universal Measuring Machine requires 
a very few simple but accurate accessories. 
In one method, an accurate, convenient 
reference edge (it may even be that of the 
workpiece) is aligned parallel to one of the 
axes. The location of this edge is found by 
using an “edge-finder.” The body of this 
gage has a 0.400 inch [10 mm in metric] 
wide slot at the top. A lower edge of the 
gage is exactly central to this slot, and is 
used to register against the given reference 
edge, Fig. 495. Once the spindle is central¬ 
ized with the slot (see Example No. 1, 
Step 11, page 291), its axis is automatically 
in line with the reference edge at a location 
which can be read on the reference scale 
and lead screw dial. Note also that when 
the indicator is set to zero at the high point 
of its contact with the edge of the slot, it is 
in effect describing an exact 0.400-inch 
[10 mm] diameter circle. 

Assume that when the reference edge has 
been located in reference to the spindle, 



FIG. 497- Shown is how the indicator is set 
as a reference to accurately sire a male 
radius. 




t-’lO. 498 —A gage block wrung onto an 

accurate lapped surface (the pick-up 
block ) provides the most accurate method 
of locating an edge for use in measuring 
size with the indicator. 




that the position recorded on this axis 
happens to be 9.7105 inches [246.647 mm], 
for example. Proof of this location and the 
accuracy of the edge-finder are accom¬ 
plished by moving the reference edge 0.200 
inch [5 mm in metric] away from the spindle 
centerline. The spindle centerline is then at 
coordinate reading 9.7105—0.200 or 9.5105 
inches [246.647 mm — 5 mm or 241.647 
mm]. The indicator should no w read zero at 
the high point of its contact with the refer¬ 
ence edge as the spindle is swung through 
an arc. 

The reference edge may be given a more 
convenient nominal setting, such as 9.0000 
inches [or 200 mm] if desired, by locking the 
axis and adjusting the reference scale and 
dial to this figure, as shown previously in 
Fig. 477. 

Once the location of the edge is recorded, 
it may be moved any specified distance 
from the spindle and the indicator linkage 
extended manually until a zero is once 
again set. The indicator can thus be made 
to describe an arc the equivalent of a highly 
accurate gage of any chosen radius. For 
example, suppose that the hole sizes of the 
test piece shown in Fig. 474 are to be 
determined. The nominal size of the hole 
is 1 inch [or 25 mm]. An arc of 0.500 inch 
radius is set by moving to the coordinate 
9.7105 inches [12.5 mm radius at 234.147 
mm] and striking a zero at the high point 
of the arc of the indicator probe against 
the reference edge, as in Fig. 496. Leaving 
the indicator fixed, each hole is next com¬ 
pared to this setting at its respective 
coordinates. If “minus,” the hole is over¬ 
size; if “plus,” the hole is undersize—by 
twice the amount shown on the indicator 
meter. 

The sizes of male radii are measured as 
in Fig. 497. The reference edge is moved to 
the opposite side of the spindle and a zero 
is struck at the low reading of the indicator 
as it is swung through an arc. Note that 
only a small portion of the male arc may 


Fin. 499 — The edge of the pick-up block 
is first tapped into alignment with one of 
the machine axes, then the block is 
clamped firmly in place. 
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Vi;;. 500 . The gage block is carefully 

uirung ivith a rotating motion to the. 
i/iek-up block. 



be used, since the indicator probe is re¬ 
stricted in rotation by the reference sur¬ 
face. 

Through use of this technique, it is pos¬ 
sible to measure the size of all radii, both 
male and female, of the piece shown in 
Fig. 494. The advantage of the Universal 
Measuring Machine for such measurement 
tasks is that location, hole size, roundness, 
lean and taper may all be measured and 
thus interrelated simultaneously. 

Although the “edge-finder” method is 
the most convenient for jig grinder or jig 
borer use, it is not the most accurate. The 
following method, using a “pick-up-block” 
(a standard accessory) is preferred. 

Example No. 6—Pick-Up Block 
Method 

The technique described here may also 
be used to locate a reference edge for the 
purpose of setting an arc of known size. 
However, it is most useful as a means of 
adjusting the indicator so that a null or 
zero is set with the indicator under the 
spindle axis. This relation of the probe to 
the spindle can be established in a matter 
of minutes once the method is understood, 


fig. sot— The axis is 'moved until by eye 
assessment the edge appears to coincide 
with the spindle axis. 



Fig. 498. However, an undue amount of 
time may be consumed if the proper system 
is not followed. 

Therefore, even though most of the steps 
are taken in a matter of seconds, the 
method is broken down step-by-step for 
completeness and clarity. 

1. Align the pick-up block with one 
axis of the machine, then clamp it, 
Fig. 499. 

2. Wring the gage block on carefully 
so that it protrudes slightly over the 
reference edge, Fig. 500. In so doing, 
take care not to misalign the pick-up 
block. 

3. Verify that the pick-up block is still 
aligned. 

4. Move the X axis while revolving the 
spindle until by eye the probe is 
centered over the reference edge, 
Fig. 501. 

5. Adjust the indicator probe tip at 
right angles to its measuring direc¬ 
tion until visually it appears to have 
no run-out when revolving the 
spindle, Fig. 502. 

6. Without altering the lead screw, face 
the probe in its measuring direction 


FIG. 502 — The indicator probe is adjusted 
at right angle to its measuring direction 
through a clutch arrangement until by eye 
assessment the probe-tip appears to have no 
run-out. The edge of the block is a convenient 
visual reference to which the probe-lip map be 
compared when swung through 180°. 



toward the pick-up block. Rotate the 
spindle while adjusting the probe 
until a zero is struck when the high 
point of the arc registers against the 
pick-up block, Fig. 503. 

The fine adjustment on the gage 
head and the zeroing feature of the 
meter will facilitate setting this null. 
Record the lead screw reading at this 
position. 

7. Raise the spindle to clear the probe; 
make a short movement with the 
cross axis to a point opposite the 
gage block, and rotate the spindle 
through 180° to register against the 
gage block face. The zero or null 
which had previously been set on 
the meter against the pick-up block 
is not altered. Instead, the lead screw 
is employed to bring the face of the 
gage block to the exact same null 
obtained on the face of the pick-up 
block, Fig. 504. The new lead screw 
reading is recorded. 

8. The location of the reference edge 
in reference to the spindle axis is at a 
location half way between the two 
recorded lead screw readings. This 
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PIG. 503— A “zero” is struck on the edge of 
the pick-up block at the high point of the 
indicator arc. 


fig. 50<t —The indicator is not adjusted 
and the axis traversed until m zero is 
struck on the back face of the gage block— 
the same reference line in space as the 
edge of the pick-up block. 


fig. 605 —Once the edge of the ■pick-up 
block exactly coincides with the 
Measuring Machine spindle axis, a zero can 
be struck on the edge of the pick-up 
block and the wrung -on face of the gage 
Mock without moving the machine axis. 
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location is determined by adding the 
two lead screw readings and dividing 
the sum by 2. 

The lead screw is set to the location thus 
determined and the axis clamped. The 
location is proved by registering against 
the faces of the pick-up block and gage 
block as before, but without adjusting the 
lead screw, Fig. 505. 

The reference edge may still be slightly 
away from the spindle axis, as shown on the 
meter. However, final precise location is 
accomplished very simply. The probe is 
registered against either the pick-up block 
or the gage block. The reference edge is 
brought by means of the lead screw to the 
mid-point of the two meter readings ob¬ 
tained on both faces, Fig. 506. 

Final approach to position should always 
be made by turning the lead screw in the 
direction shown by the arrows on the lead 
screw dials to take out backlash. 

The procedure of locating a reference 
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edge under the spindle has been described 
in detail because it is a useful technique to 
understand for anyone interested in mea¬ 
surement. In actual practice, an edge of the 
pick-up block can be initially centered 
beneath the spindle closely enough by eye 
so that the deviation is within the full 
meter range at its lowest magnification, 
which is ±0.010 inch [ ± 0.2 mm in metric]. 
In this case, the setting of the lead screw is 
established as simply the mid-point of the 
deviation of the pick-up block and gage 
block as shown on the meter. Final setting 
is then accomplished by switching the 
electronic indicator to a higher magnifica¬ 
tion, such as 20,000X, and setting to the 
mid-point once again. 

A special attachment which fastens to 
the front of the housing, Fig. 507, enables 
the probe to be automatically positioned 
in line with the spindle to within 0.0001 
inch [0.0025 mm]. Once “on the meter,” 
final precise zeroing of the indicator is ac¬ 
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complished with the pick-up block and 
wrung-on gage block. 

Absolute Measurements 

Once the reference edge has been located 
under the spindle centerline and a zero 
set against this edge, a variety of absolute 
measurements may be performed. Ex¬ 
amples may be shown by referring once 
again to the demonstration piece of Fig. 
474. The length of the piece is measured 
by striking a zero indicator reading at one 
end and recording the lead screw setting. 
The spindle is rotated 180° and the table 
is traversed until the other end registers 
a zero on the indicator probe. The length of 
the demonstration piece is equivalent to 
the difference in lead screw readings re¬ 
corded at either end, Fig. 508. The 4-inch 
width [100 mm in metric] of this piece is 
measured in the same manner. 

Size of the holes may also be measured 
with this technique. Once the location of 
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FIG. 606 — Final precise alignment of the 
edge of ike picl-up block with the machine 
spindle is accomplished by noting the 
reading-obtained against the pick-up block 
and the gage block without having moved thr¬ 
eats. Exact location of the edge is the 
mid-point of the two readings. 


fig, 607 —A special attachment which 
fastens to the front of the quill housing 
allows the indicator probe to be sal on the 
spindle axis within ,0001 inch [0.0025 mm] 
for fader alignment. 


® Pick-up block 
(§) Mid-point 
(c) Gage block 



the holes is determined, their centerlines 
are also known for the purpose of measur¬ 
ing size. 

It should now be apparent that with this 
method it is possible to measure to any 
point in the 11 by 18 inch [280 mm by 460 
mm] plane of travel of the machine. 

Measure Diameter of Ring Gage— 

3 Methods 

In order to measure the diameter of a ring 
gage, or any hole, its centerline must first 
be determined, and is accomplished by any 
one of three different methods.* The first 


*At the time of writing, a new “sizing” probe 
has become available. The design was arrived at by 
cooperation of engineers at Federal Products Corp. 
and the Moore Company, specifically for use with 
the Moore Universal Measuring Machine. The use 
of this probe will allow sizes to be measured with¬ 
out first zeroing on the pick-up block and without 
rotating the spindle. Most importantly, probe-tip 
size still does not enter into consideration . 




fig. 608 —The demonstration piece is used 
here to show how the Universal Measuring 
Machine may be used to perform a 
variety of absolute measurements . All these 
measurements may be oxeomplished once a 
“two” is set with the indicator on an edge 
directly on the spindle axis , 
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Fit;, |>09 . (fop) One method of measuring the 

size of a hole is to Irani, one axis hack and 
forth with the indicator registered in the 
hole so as io find the “low” point which is 
■its centerline. Size of the hole is then 
measured on this centerline with the other 
machine axis. 


FIG. 61 . 0 -. {center) A more exact method of 

determining the centerline of the hole for 
purposes of measuring size with a set 
probe is to strike a “zero” by rotating the 
spindle at locations A and S using one 
machine axis. The centerline is exactly 
between the two lead screw readings obtained. 



“Zero” low reading 
at 5.03240 
coordinate 




(C) (D) 
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method, in which the probe is extended 
to register against the sides and the lead 
screw is used to center the piece under the 
spindle, has already been described in Ex- 
ampleNo. 1,Step 11,page290. Thismethod 
is the most accurate way to find the center 
of the hole but this accuracy is unneces¬ 
sary and moreover time consuming when 
only size is to be determined. Also it is more 
convenient to be able to determine the cen¬ 
terline of the hole without having to alter 
the probe, once a zero has been set against 
the pick-up block. 

The second method, illustrated in Fig. 
509, does not require altering the probe. 
One axis of the machine is traversed back 
and forth while the indicator is registered 
normal to one side of the hole. The center- 
line of the hole is at that point of the axis 
where the lowest indicator reading is re¬ 
corded. The hole is then measured with 
the other axis along this centerline from 
C to D. This is by far the fastest method. 
While it does not establish the centerline 
of the hole as precisely as the previous 
method, the error which may be introduced 
through its use is negligible except for 
small holes. 

A third method, illustrated in Fig. 510, is 
the most precise for determining the cen¬ 
terline of the hole, while still leaving the 
probe set to zero under the spindle axis. 
This method is preferred for orienting 
single small holes. The steps are: 

1. Strike a zero by rotating the spindle, 
with probe contact at a point such as 
A. Record the lead screw reading. 

2. Similarly, strike a zero at point B 
directly opposite, moving only the 
cross axis. 

3. Halfway between A and B is the 
centerline of the hole. Its diameter 
may be measured with the long axis 
in the manner previously described. 

Since the centerline of the hole in the 
other direction is halfway between the 
coordinates at C and D, a second measure- 


FIG, 5U — Size of a hole, such us a ring 
gage, may be measured at any depth 
simply by employing the vertical 
adjustment of the measuring machine quill. 
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F 1 <S, 512 --(top) All aspects of hole geometry, 
as with this ring gage, may be determined 
through use of the accurate movements built 
into the Universal Measuring Machine. 


fig. 513— (center) Concentricity of top to 
bottom of a hole is determined by making a 
polar tracing, adjusting the quill lo a new 
depth and then comparing the two tracings. 


ment of the hole may be made across its 
diameter using the cross lead screw. This 
second check is valuable since it provides a 
measurement of the hole across two diam¬ 
eters, and to some extent double-checks 
the first measurement by comparing the 
two lead screw measurements. 

The diameter of the ring gage at the 
bottom is measured by simply lowering the 
spindle and traversing from C to D as 
in Fig. 511. 

All the sides of the ring gage may now 
be probed to determine straightness, lean 
and other geometric aspects of the hole, 
Fig. 512. 

While the ring gage is in the same posi¬ 
tion, a polar recording of roundness may be 
made. The concentricity of the top and 
bottom of the hole can also be determined, 
Fig. 513. The reader should also refer to 
the previous section (see pages 268-272) for 
further information on using the Universal 
Measuring Machine for measuring round¬ 
ness. 

Internal comparators are the usual in¬ 
struments for the measurement of hole size. 
These instruments usually consist of two 
probes which project vertically from a 
platen which supports the piece to be 
measured. The two probes may be set to 
read differentially, as in Fig. 514. In this 
case, the reference size is a stack of gage 
blocks or a master ring gage to which the 
hole size is compared. This design usually 
allows greater accuracy in determining 
hole size than other instruments which 
have built-in length-measuring systems. 
The latter instruments, however, avoid the 
need for masters or stacks of gage blocks. 

It is not meant to imply that the Univer¬ 
sal Measuring Machine surpasses such 
comparators. Internal comparators, es¬ 
pecially the differential type, can hardly 
be excelled for speed and accuracy in 
repetitive-type measurements. The ad¬ 
vantage of the Measuring Machine for 
such applications is that it allows the 




Ring gage 



fig. 514 . Very accurate determinations of 

ring gage size are made by internal 
comparators which have been set against 
gage blocks . 
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FIG. 015 —The ten ring gages shown are in 
10 millionths of an inch increments 
[0.00025 mm] and art used as a physical 
test of size of the rotary table spindle 


assembly. The Universal Measuring 
Machine allows size, roundness and other 
geometric characteristics of the holes to be 
inspected in one set-up. 




measurement of “specials,” especially 
large, odd size holes when there is no 
available master to which the hole may be 
compared, or when there is doubt as to 
the accumulated accuracy of a number of 
gage blocks that have been wrung together. 
It has been the author’s experience that 
the wringing films may vary surprisingly 
in thickness, in actual workshop practice. 
It is therefore standard practice at Moore 
to measure master ring gages with the 
Universal Measuring Machine. 

An example of th use of the Universal 
Measuring Machine for calibrating the 
hole size of masters is shown in Fig. 515. 
The ten ring gages being measured differ 
in size by increments of 10 millionths of an 
inch [0.00025 mm] for a total spread of 
0.0001 inch [0.0025 mm]. These gages are 
used as a final physical test of the assembly 
size of the rotary table spindle. Since they 
are used in production and are subject to 
wear, the ring gages are calibrated on a 
calendar basis as shown for size, roundness 
and straightness. 

THE MODEL No. 1V 2 
UNIVERSAL MEASURING 
MACHINE 

The No. lRj Universal Measuring Ma¬ 
chine, which is shown in Fig. 516, has vir¬ 
tually the same principles of construction 
and measurement capabilities as the No. 3 
model. The essential differences are the 
lesser capacity of 9 by 14 inch travel 
[230 by 355 mm] and the use of scraped 
cast-iron V-ways, rather than the hard¬ 
ened, ground and lapped ways of the No. 3 
model. Although initially made to the 
same accuracies, the wear-life of the No. 
1 model is therefore much less than that 
of the No. 3 model. 

The smaller model has been made avail¬ 
able to allow small workpieces to be 
measured with a lesser investment by the 
owner. For this reason, this model is an 


fig. 510 —The No. 1% Universal 
Measuring Machine employs the same 
design characterislics as in the No. 3 
Universal Measuring Machine, but has a 
lesser travel of 9 in. by 1] in. 


[230 r 355 vim in metric machines]. Also, 
maling way surfaces are of scraped cast 
iron and arc not hardened, ground and 
lapped as in the No. 3 model. 
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pit;, at v . 'This model No. 1*/% Measuring 

Machine is shown with a built-in read¬ 
out ipiioie accuracy still derives from the 
lead screw. The readout may be used to 
record radius size of the ball track (shown), 
and the lead screw dials to record 


simultaneous other part, dimensions. 
The readout may be switched on and off 
as often as desired without accumulated 
loss of accuracy, since the lead screw 

dials, once set, provide a permanent, 
reference. 


ideal selection when the essential applica¬ 
tion is to be the measurement of roundness 
of small parts. 

READOUT ON THE 
UNIVERSAL MEASURING 
MACHINE 

An example of the measurement of small 
parts on the Model No. 1J4 Measuring 
Machine is shown in Fig. 517. This par¬ 
ticular machine has been fitted with a 
“Vu-point readout.”* Virtually all of the 
accuracy of the lead screw is retained when 
a readout is adapted to any model of the 
Universal Measuring Machine, since en¬ 
gineering practice is to use only rotary 
transducers to monitor the indexing of the 
lead screw. 

A great advantage of the readout, as 
used on a Universal Measuring Machine 
which measures displacement by means of 
an accurate lead screw, is that two separate 
types of dimensions may be recorded simul¬ 
taneously with only one measuring ele¬ 
ment. For example, the reference scale and 
dials may be used to record coordinate 
locations, while the readout is used to re¬ 
cord individual hole size, (or vice versa). In 
Fig. 517, the readout records the cross- 
section size of the ball track, using the 
pick-up block to the left to set the indicator 
to the given radius. The reference scales 
and dial may be used simultaneously 
to record the circular diameter of the ball 
race from the center of its track radius as in 
Fig. 518. 

The readout may be switched on and 
off or re-set to new figures as often as de¬ 
sired during the measurement process with 
no loss of accuracy, since the lead screw 
dial settings provide a permanent refer¬ 
ence. 

The readout package may also be fur- 


*Vu-point readout manufactured by Remex Elec¬ 
tronics, Division of Ex-Cell-0 Corporation. 



Diameter 

from center of radius 
recorded with 
lead screw dials 


u 

Radius recorded / 
on readout f 


Ball bearing internal race 



fig. 518 . Drawing of the internal race 

shown in Fig. 517 demonstrates the 
convenience of using the readout and the 
lead screw dials simultaneously. 
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FIG, hui—Thc ability of the Universal 
Measuring Machine lo perform 
“absolute measurements” is particularly 
essential in the measurement of cam 
shapes, as with this master for an 
automotive cam shaft. 




nished with a printout in order to record 
all dimensional values. 

Cam Shapes 

Fig. 519 shows an automotive master cam 
mounted on the Measuring Machine and 
rotary table. The cam has beei generated 
by a tape-controlled jig grinder. 

The value of being able to measure ab¬ 
solutely, once the probe is centered with 
the Measuring Machine spindle, is partic¬ 
ularly apparent for such applications. Cam 
shapes may be measured at as many tens or 
hundreds of points as is feasible or prac¬ 
tical. Measurement may be performed en¬ 
tirely by rectilinear coordinates or by a 
combination of angular rotation and linear 
movement as shown here. 

3. Applications of the 
Universal Measuring- 
Machine 



The basic techniques of measuring size, 
roundness, and rectilinear and polar coor¬ 
dinates through use of the Universal 
Measuring Machine have now been de¬ 
scribed. With a little imagination, these 
same techniques may be applied to solve 
an extremely wide range of measuring 
tasks. It is instructive to present some 
typical measuring applications which dem¬ 
onstrate the machine’s even greater versa¬ 
tility when used in conjunction with other 
available instruments. 

USE OF ANGULAR-MEASURING 
INSTRUMENTS ON THE 
UNIVERSAL MEASURING 
MACHINE 

The Rotary Table 

The rotary table is the most versatile in¬ 
strument for a wide range of measuring 
applications where angle is the essential 
consideration. 


FIG. 520 --Angular spacing of gear teeth is 
most truthfully inspected when the Rotary 
Table is used in conjunction with a 
Universal Measuring Machine rather than 
by surface plate methods. Use of a gage- 


pin allows both flanks of the gear tooth as 
well as center-distance to be included in the 
measurement. Two gage pins aid in 
initial aligning of the gear. 
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V'ig, 521 . The rotary table is shown 

mounted on the Universal Measuring 
Machine, utilizing its built-in right angle 
feature. The thin walled aluminum pan 
has been permanently fix In red until till 


ptacisc m-aitffiring and inspection arc 
completed. Holes in this pari must he 
positioned, and aligned within .0001 inch 
\().()Odh mm}. 


Used Flat 

In Fig. 520, the tooth-to-tooth spacing of 
a gear is inspected through use of the 
rotary table. The gage pin is of a specific 
size to establish its axis on the pitch line 
of the gear, and is placed in each consecu¬ 
tive notch as the rotary table indexes to 
the specified angle. 

Such an inspection is usually performed 
on a surface plate, the gear being mounted 
with its axis horizontal on some form of 
indexing head. Angular spacing of the teeth 
is then determined with an indicator and 
stand, using the high point of the pin as a 
reference. 

However, the angular spacing of the 
teeth is more truthfully represented by 
picking up the axis of the pin, Fig. 520, 
rather than its high point. The center-dis¬ 
tance of the teeth, an important dimension, 
can also be determined simultaneously. 

The use of two pins at opposite diam¬ 
eters as shown insures the most accurate 
orientation of the gear to the measuring 
system. The technique is to first align both 
pins with the measuring machine axis, 
while intersecting the rotary table rota¬ 
tion axis (if the gear has an even number 
of teeth). Re-checking the alignment of the 
pins after turning the rotary table through 
180° provides a sensitive proof, since any 
set-up error is doubled. 

Used Horizontally 

In Fig. 521, the rotary table is mounted 
with its axis horizontal, utilizing its built-in 
right angle feature. Shown is a fragile, thin- 
section aluminum part. To establish an 
adequate reference and clamping surface, 
the part is permanently fixtured on the 
flat plate shown until final machining and 
inspection are completed. The alignment 
of the holes in this part, which is partic¬ 
ularly critical, may be accurately measured 
because of the exceptional angular and 
rotational accuracy of the Ultra-Precise 
Rotary Table. 
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■i’lG. 522 —A special accessory for the 
lioiary Table is the Tailstock Unit; 
shown here, it is used to make possible an 
inspection of lead and periodic error of « 
thread-ground worm. 
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mounted between center? of the Rotary 
Table and Tailstock. Geometry, radial size 
and angular spacing of the blades may be 
inspected in this manner. 


KlG, 524— A. pT0Ct& 
inics ts mounted oh 
centers on the Rota 
A graticule in the ? 

Microscope is used to position each 
hue io (i reference line, included angU 
being determined, with the rotary table 


Used With Tailstock Unit 

A tailstock unit is available for use with 
the rotary table to allow the measure¬ 
ment of features of a part when held be¬ 
tween centers. In addition to the tailstock 
itself, this unit consists of an adjustable 
center which mounts on the face of the 
rotary table. When thus equipped, the 
rotary table may perform the functions of 
an indexing head, but with the additional 
advantages of a large swing of inches 
[393.7 mm], accuracy to ±2 seconds, and 
the option of either reading angles on the 
dial or using an index plate (see “Dividing 
the Circle,” p. 218). 

In Fig. 522, the tailstock has been re¬ 
moved from its storage nest in the mahog¬ 
any rolling cabinet (foreground) and 
mounted on the machine table. The part 
which is being mounted will be inspected 
for overall thread lead, and indexed by 
use of the rotary table to measure periodic 
error. 

In Fig. 523, a turbine has been arbor- 
mounted between centers. As each blade 
is indexed over center, its size, geometry 
and angular spacing are compared. 

In Fig. 524, a 500-line dial is mounted 
on an arbor and inspected with the stan¬ 
dard 30-power microscope. 

The Precision Index Center 

The Index Center shown mounted on the 
Universal Measuring Machine in Fig. 525 
also allows the inspection of parts held 
between centers. Spindle accuracy of this 
instrument is better than 15 millionths of 
an inch [0.00038 mm] T. I.R. Indexed posi¬ 
tion is governed by hardened, jig-ground 
index plates, accurate to ±6 seconds, 
which are fastened to rotate integral with 
the Index Center spindle. The knurled 
knob in the inspector’s right hand rotates 
the spindle, and the carbide pin in his left 



pig. 626 —The Precision index Center 
provides a rapid method of angular 
indexing of parts. It is designed with a low 
profile for convenience in inspecting 
smaller patls. 
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PIG. 526 — When roundness of a conical 
■part is inspected using a leveling plate, 
the axis of the cone may unknowingly he 
shifted to obtain the best circular trace. 
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is used to locate from a lapped carbide hole 
to the index plate hole. 

Because of its fast indexing capability 
and its low profile, the Index Center is 
ideal for repetitive-type indexing and for 
the inspection of small gears and gear dies. 
This instrument is also designed to allow 
through-grinding on the surface grinder, 
and is sealed to make wet-grinding appli¬ 
cations possible. 

The Micro-Sine Table 

The No. 3 Micro-Sine Table is particularly 
adaptable to measuring machine use since 
it is capable of great accuracy when set 
with care by the inspector (see page 214). 

One of its unique advantages when used 
with the Universal Measuring Machine 
is illustrated in Figs. 526 and 527. Conical 
parts are typically inspected for roundness 
as in Fig. 526, using the leveling table to 
obtain the best circular trace if the piece 
has no reference surface to use in align¬ 
ment. However, in doing so, the axis of the 
cone may be unknowingly adjusted out of 
alignment to compensate for its roundness 
condition. When the conical part is 
mounted on the Micro-Sine Table instead, 
its angle and roundness may be inter-re¬ 
lated. Roundness is inspected while the 
Micro-Sine is at 0°, and the angle is in¬ 
spected as shown in Fig. 527. 

Rotary Table Mounted on the 
Micro-Sine Table 

The No. 3 Micro-Sine Table has been de¬ 
signed especially for use with the rotary 
table for the inspection or generation of 
compound angles. In Fig. 528, the rotary 
table and Micro-Sine Table are used to 
inspect the angular spacing of thirteen 
holes which have been bored at a 45° angle 
to the axis of the part. Note the use of the 
index plate, which facilitates the measure¬ 
ment of inconvenient angular spacings 
(such as thirteen). 

The twelve holes about the center of the 


PIG. 527 —The roundness and included 
angle of conical parts should, if possible, 
be inter-related in an inspection. This can 
be accomplished through use of the round¬ 
ness inspection features of the Universal 


Measuring Machine and the Micro-Sine 
Table. Included angle (and orientation of 
the axis ) is being determined here prior to 
performing a roundness trace. 

















FIG, 528 —The rotary table is mounted 
on the Micro-Sine Table for the inspection 
of lit-hole spacing. The index plate aids in 
the setting of such inconvenient angular 
spacings. The 12 holes about the center •/ 
the part are inspected by reading the dials 
while the Micro-Sine Table is horizontal (0°) 


part are inspected while the Micro-Sine 
is at 0°, and their angle-spacing read on the 
rotary table dial. 

1440 Small Angle Divider 

The highest grade of rotary table offered 
by Moore is accurate within ± 2 seconds. 
Although this accuracy is certainly suffi¬ 
cient for even the finest class of laboratory- 
type use, it is not truly consonant with the 
accuracy of the Universal Measuring 
Machine. To illustrate, on an 8-inch 
[203 mm] diameter workpiece, a 2-second 
error is equivalent to approximately 40 
millionths of an inch [1 pm]. In contrast, 
the Measuring Machine would have no 
more than 20 millionths of an inch [0.5 pm] 
error in this distance. Moreover, circle- 
dividing instruments in use have the un¬ 
fortunate characteristic of doubling small 
errors in alignment or in centering of the 
workpiece. 

The 1440 Index by itself is not con¬ 
venient for general use with the Universal 
Measuring Machine, since it does not allow 
for fine increments of angular movements, 
which are necessary to orient holes central 
to the Measuring Machine spindle. Also, 
this instrument is restricted to only 1440 
positions. In order to overcome the dis¬ 
crepancy in the accuracy attainable with 
linear as compared to angular measure¬ 
ments, the 1440 Small Angle Divider has 
been adapted for use with the Universal 
Measuring Machine. This instrument will 
discriminate angular values as fine as 0.1 
second. 

In Fig. 529, an index plate for use with 
the Precision Index Center is inspected 
through use of the 1440 Small Angle 
Divider. In such applications, if the 
number of holes in the piece is a factor of 
1440, only the 1440 index component is 
indexed, and the accuracy of angle-setting 
is +0.1 second. At all other numbers of 
hole-spacings, accuracy is within ±0.5 
second. 



FIG. 529 —The most accurate instrument for 
resolving angular errtrs with the Universal 
Measuring Machine is the 11,1,0 Small 
Angle Divider. One. thousand, four 
hundred and forty increments are inspected 
within ±0.1 second. This instrument will 


set. to increments as Jine as 0.1 second 
■with an accuracy of ±0.5 seconds over 
full 12,900,000 increments. Under 
inspection is a jig ground index plate 
used with the Precision Index Center. 








The inspection of geometric forms to a 
high degree of accuracy is possible when 
the 1440 Small Angle Divider is mounted 
on the Measuring Machine. In Fig. 530, 
the 90° included angle of a master V- 
block is measured. The procedure is to 
first align one side of the V with the 
machine axis while the 1440 Small Angle 
Divider is at 0°. Next the other side of the 
V is rotated into alignment with the ma¬ 
chine axis by use of the 1440 Small Angle 
Divider. The included angle is read di¬ 
rectly. By extending the quill, the included 
angle and the shape of each side is inspect¬ 
ed along the full length of the V-block. 

Angular relationships in another atti¬ 
tude are inspected by mounting the 1440 
Small Angle Divider on a special angle 
iron, as shown in Fig. 531. The slots and 
holes of a component of a film-cutting die 
are inspected by mounting the 1440 Small 
Angle Divider with its axis horizontal. 

Use of the Strip Recorder 

A linear recorder, Fig. 532, may be used to 
record indicator values. Vertical magnifi¬ 
cations of the trace are set on the meter 
panel. 

This accessory has many possible appli¬ 
cations such as the recording of roundness, 
straightness, angular deviations or errors 
of surfaces of a workpiece from their 
specified coordinates. The method of 
calibrating the master straightedge against 
the machine axis shown in Fig. 532 is 
ideal, since any deflection of the straight¬ 
edge in this attitude is in a neutral direc¬ 
tion to the surface being measured. Devia¬ 
tions from straightness are shown on the 
graph (right). 

Straightness of the machine axis is self- 
proved upon turning the straightedge over 
and making a second tracing of the same 
side. 

Fig. 533 shows the same principle ex¬ 
tended for an even more comprehensive 
calibration. Attached to the housing on a 










common bracket are two indicator gage 
heads. Switching facilities built into the 
meter-panel allow the readings of either 
probe (or even a third probe) to be viewed 
on the meter, or to be recorded. The upper 
trace on the chart is a recording of the 
straightness of the far edge of the straight¬ 
edge using probe No. 1. The lower trace is 
of the front edge of the straightedge using 
probe No. 2. The center trace of the chart 
records the variation of thickness of the 
straightedge, obtained by operating the 
two probes differentially. 

SPECIAL MACHINES 
EMPLOYING THE No. 3 OR 
No. 1V 2 BASE CONSTRUCTION 

The coordinate-system is a fundamental 
mechanical principle which is applied to a 
countless number of tasks in science and 
industry. The No. 3 base construction is a 
coordinate-system which has been proven 
to be capable of the ultimate in accuracy. 
Yet it is made in sufficient quantities 
(several hundred a year) to enable econom¬ 
ical production. It is therefore a logical 
choice for other applications requiring 
slide-way movements, but to jig borer or 
measuring machine accuracies. In addition 
to those uses already described (see pages 
223 and 235), the No. 3 base has been ap¬ 
plied as the coordinate system for such 
diverse applications as electrical discharge 
machining (Keystone Carbon, Eltee Man¬ 
ufacturing), micro-circuit analysis (CENG, 
Grenoble, France), fresnel lenses (Alliance 
Tool and Mold Corp.) and a ruling engine 
for diffraction gratings (Massachusetts In¬ 
stitute of Technology).* 

Recently, the No. 3 base slide-way has 
been applied with a roller way construc¬ 
tion, Fig. 534. The advantage of this design 

*For more detailed information see: George R. 
Harrison & Stephen W. Thompson, “Large Diffrac¬ 
tion Gratings Ruled on a Commercial Measuring 
Machine Controlled Interferometrically,” JOSA, 
Vol. 60, No. 5, 1970. 


FIG. 532 -The Strip Recorder allows 
indicator valves to be recorded on a strip 
chart. This instrument has many applica¬ 
tions, one of which, the recording of 


straightness of a master straightedge, is 
shown. The straightedge is rested in a 
neutral direction to bending, and may be 
reversed to self-prove the machine. 



FIG. 533 —Three features of the master 
straightedge are recorded on the strip 
chart. The top line slums straightness of 
the far side of the straightedge, the bottom that 
of the near side. Tim middle tine records 
variation in straightedge thickness, using 
the indicators set up differentially. 
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is that the limitation imposed on accuracy 
by the accumulated effect of friction, oil- 
film, and “wind-up” of machine members is 
virtually eliminated. Repeatability and 
sensitivity of movement to fractions of a 
millionth of an inch [hundredths of a 
fim| are possible with this design (see page 
320). Examples of applications of this 
design are as a micro-densitometer (Ansco), 
Continuous Path Jig Grinder,* fabrica¬ 
tion of integrated circuits (Stewart-Warner 
Company), grinding of aspheric lenses 
(Bell and Howell Company, Canon Cam¬ 
era Co.) diamond-tool facing lathe (Law¬ 
rence Radiation Laboratory, University of 
California), optical scanning machine (U.S. 
Geological Survey), artwork generator 
(Concord Control, Inc.).** 

4. The No. 4 and No. 5 
Universal Measuring 
Machines 

The No. 4 Universal Measuring Machine 
has an 18 by 32 inch travel [460 by 800 mm], 
and the No. 5 Universal Measuring Ma¬ 
chine has a 24 by 48 inch travel [600 by 
1200 mm.] Both are of a planer-type con¬ 
struction. A third Z axis is available in 
either model and is demonstrated in the 
model 4Z Universal Measuring Machine in 
Fig. 535. The No. 5 Universal Measuring 
Machine shown in Fig. 536 has been 
adapted for automatic measurement by 
numerical control (Union Carbide Nuclear 
Company). 


*For more detailed information see: Rupert 
LeGrand, “N/C Jig Grinder Makes Molds,” Ameri¬ 
can Machinist, January 29, 1968; Richard C. Fer¬ 
guson, N/C Jig Grinder Joins the Ranks,” Modern 
Machine Shop, February, 1968; William Hoffman, 
“Diemakers Big Step—N/C Contour Jig Grinding,” 
Tooling & Production, March, 1970; “Numerisch 
gesteuerte Koordinaten-Schleifmaschine, Werkstatt 
und Betrieb, 1968. 

**“The Concord Artwork Generator: A High- 
Precision Automated System for the Production of 
Integrated Circuit Artwork Masters,” Publication 
of Concord Control, Inc., Boston, Massachusetts. 











PIG. 535 — Thu model No. /+ Measuring 
Machine has an .18 in. In/ .82 in. travel 
\I).6Q mm x 800 mm], A vertical Z axis 
shown on this machine, can be fitted to any 
model of Universal Measuring Machine. 


Considerations of size, weight and in¬ 
tended function have dictated the selection 
of the planer-type design for these ma¬ 
chines. A compound-type design as used in 
the No. 3 model would be unnecessarily 
huge and cumbersome to attain this 
capacity for purely measurement purposes. 

For maximum accuracy, the rail of these 
larger measuring machines does not have a 
means of elevation, normally provided 
in the planer-type jig borer. Instead, up¬ 
rights are cast integral to the base for 
maximum strength, and to eliminate 
stresses when the bridge is bolted to the 
base. Riser blocks are used between the 
base and bridge in order to meet the 
user’s specifications as to clearance under 
the bridge. 

The base carries the table, or X axis 
only, the Y axis traveling on the bridge 
overhead. Because of the greater masses to 
be moved (the table of the No. 5 model 
weighs 1000 pounds [453.6 kg], the mov¬ 
able members are supported on rollers. 
This design virtually eliminates loss of ac¬ 
curacy which may occur from frictional 
heat and “stiction.” 

The planer-type jig borer normally uses 
as guide ways a V at the top of the rail, and 
a flat guide at the lower front face of the 
rail, which is the proper design when there 
is metal-cutting to be performed. However, 
the design of the No. 4 and No. 5 models, 
not having to consider the stress of cutting, 
or of machine wear, or friction, can em¬ 
ploy a double-V construction for maximum 
accuracy. 

All machine members are of close- 
grained Meehanite iron for stability and 
are strongly ribbed for rigidity. 

As in all Moore Universal Measuring 
Machines, a l/10th inch pitch modified 
acme thread lead screw is employed [3 mm 
pitch in metric models]. Because of the 
greater lengths of the lead screws in these 
models, precision rollers are used to sup¬ 
port and align them as shown previously 



pig. 530 —The No. 5 Universal Measuring 
Machine unth Numerical Control. 
Numerical Control is readily adaptable 
with many possible combinations to a 
coordinate machine whose accuracy 


derives from a precision lead screw. 
Numerical Control is especially suited for 
the inspection of templates or cams wherein 
many minute incremental movements are 
to be made and recorded. 
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in Fig. 329. This design also minimizes 
frictional heat of the lead screw in use. To 
further maintain the temperature of the 
lead screw, it is made to operate in a res¬ 
ervoir of oil. The oil in this reservoir con¬ 
tinually flows down into the base of the 
machine, where any slight build-up of heat 
is transferred to the large mass of the base. 
The oil is next pumped back into the lead 
screw trough. 

To avoid a dead air space in the cavity 
of the base, a fan is used to cause through- 
flow of air, which helps to maintain the 
ambient temperature of the casting as well 
as the circulating oil. 

Although on the one hand there is a con¬ 
siderable overall gain in accuracy by sepa¬ 
rating the axes through a bridge-type con¬ 
struction, this design has the unfortunate 
effect of making the temperature problem 
more acute. It is now more essential that 
temperature be exact and that stratifica¬ 



tion be prevented. Since it is virtually im¬ 
possible to completely eliminate errors due 
to temperature stratification (vertical gra¬ 
dations in ambient temperature) within a 
room environment, the bridge axis is spec¬ 
ified to no closer tolerance than that of 
the base axis, despite the fact that it is of 
a shorter travel. 

In addition, when more clearance is 
given under the bridge by means of a riser 
block, the tolerance must be broadened. 
The need for this is attributable not so 
much to difficulty of construction but more 
to the increased problem of stratification 
as the axes are further separated. 

Despite the large masses to be moved, 
the repeatability attained by this design is 
exceptional. In an experiment employing a 
non-contact “fiber-optic probe,” it was 
seen that the machine could make repeat 
settings within 1/5 millionths of an inch 
[0.005 pm], and could also move in discrete 
increments of that small an amount when 
the eye was aided by a microscope in read¬ 
ing the lead screw dials.* 

In later models of No. 4 and No. 5 Mea¬ 
suring Machines, adapted for special pur¬ 
poses, a master rotary table gear and worm 
have been fitted to the end of the lead 
screw shaft, together with other reduction 
gearing to provide tremendous reductions 
of up to 100,000 to 1.** When a relatively 
simple stepping motor is placed at the end 
of the gear train, it is found that the ma¬ 
chine table can be pulsed automatically 
through such small increments. 

The means by which such small displace¬ 
ments are measured is shown in Fig. 537. 
To the left are the controls which deter¬ 
mine the particular number of pulse incre¬ 
ments of the stepping motor. In the fore- 


*Wayne R. Moore, “Repeatability to 0.2 mil¬ 
lionths!”, American Machinist, August 29, 1966. 

**To arrive atthis figure, the author used a some¬ 
what loose interpretation of reduction, but neverthe¬ 
less a fair one: a 0.050 inch movement of a 3 inch dial 
used in place of the stepping motor would displace 
the table only 0.0000001 inch. 


f’S|, 538— light-gap of the fiber-optic 
probe to the reflecting surface is made to 

ad on a vlado-cell, then in electronically 
amplified to appear on an oscilloscope 
screen.. Separation of the lines is JjOQ itu 
{Uiviiit} and represents a w.awdm® 
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movement of t millionth of an inch 
[1/i f 0th, pin}. Roller-way measw ing 

machines have been seen to make ten 
distinct 'movements within this i niiiiio 
inch [1/fOih §[»;/], 
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■a-t ask ring machines and ruling engines. 


1. Left foreground—Ruling Engine built for Professor 
George W. Stroke of the State University of New 
York—to produce diffraction gratings. 

2.48-inch square master surface plate. 


3 . 3rd axis assembly of the Moore No. 4Z Universal 
Measuring Machine. 

4 . Model No. 4Z (3 axis) Universal Measuring Ma¬ 
chine built for International Business Machines 
(IBM). 

5. Universal Grid and Scale Calibration Machine, 
built for Bausch & Lomb. Calibrations are based 
on direct laser measurements. 

6. Test set-up for automatic calibration of Rotary 
Tables. 

7 . In glass cabinets—Moore Master Lead Screws of 
various lengths to be used in Measuring Machines. 

8. Part of laser system (Spectrophysics) to be used 
with Ruling Engine for Professor George W. Stroke. 

9 . Slo-Syn Controls for automatic stepping of Lead 
Screw to as small as 0.2 millionths movement 
(used on Professor George W. Stroke’s Ruling 
Engine). 

10 . 42-in. Surface Plate on which rests various ele¬ 
ments that make up diamond-ruling mechanism, 
including diamond-lifter, straight-edge, monorail 
and diamond carriage. 


fig. h-\o-- Vari(lij of projects, underway in 
the Mensuring Machine Laboratory. 
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4?ru,, ©41 —The Jf.8 inch [ t ?A!) mm] box-type 
surface plate, used as a master, is also 
convenient as a datum to ins-pert heavy 
machine parts and assemblies. Shown is the 


inspection of a Z axis assembly of the 
No. fZ Universal Measuring Machine 
( background ). 



ground, the fiber-optic probe is mounted 
on a box parallel with the light-discharge 
end held in proximity to a reflecting sur¬ 
face fastened to the machine table. The 
intensity of the reflected, returning light 
varies with the distance of the light probe 
to the reflecting mirror. The varying light 
intensity acts on a photocell contained in 
a meter (to the rear of the machine table). 
The varying voltage is amplified to be 
viewed on the oscilloscope screen to the 
rear. 

A close-up of the oscilloscope screen is 
seen in Fig. 538. The separation of the 
lines shown is a distance of 0.400 inch 
[10 mm] and represents a machine move¬ 
ment of 1 millionth of an inch [l/40th pm]. 
The lead screw, when pulsed, can be seen 
to displace the machine table in ten dis¬ 
tinct movements within this 1 millionth of 
an inch [l/40th pm]. 

The ability of these larger Measuring 
Machines to move in such fine increments, 
together with their geometric accuracy, 
has led to their being adapted to many 
different uses, apart from that of purely a 
measuring function. 

To fulfill these expanding requirements, 
a separate laboratory has been constructed 
expressly for the purpose of calibrating and 
assembling the larger measuring machines, 
Fig. 539. Some of the projects undertaken 
and equipment shown in use in Fig. 539 are 
illustrated in Fig. 540. 

In Fig. 541, a Z axis of the No. 4 Model 
Measuring Machine is being assembled 
and inspected while mounted on a master 
48 inch [1219 mm] surface plate within this 
laboratory. 


fig. 542 —The No. 5 Universal Measuring 
Machine has ample capacity to perform 
calibrations of end standards and line 
standards of considerable length. Shown 
is the inspection of a line standard -using a 
SOX microscope -mounted in the Measuring 
Machine spindle. 
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fig, 513 . The granite surface plate has 

been adjusted parallel to the plane of 
travel •/ the Universal Measuring 
Machine axes through Ike use of steel 


• wedges, Calibration readings may be taken 
directly and require no calculations, os is 
normally the case with an autoeollimator- 
type inspection of a surface plate. 


APPLICATIONS OF LARGE 
UNIVERSAL MEASURING 
MACHINES 

Measuring techniques and applications of 
the standard No. 4 and No. 5 Universal 
Measuring Machines are virtually identical 
to that of the No. 3 model. A few applica¬ 
tions are included here to indicate the 
greater capacity and versatility of these 
machines. 

In Fig. 542, a master precision scale is 
inspected while mounted on a No. 5 Uni¬ 
versal Measuring Machine. 

A granite surface plate has been leveled 
by means of steel wedges in Fig. 543, so 
that its top surface is parallel to the plane 
of travel of the Measuring Machine. Mea¬ 
surement of flatness of the granite surface 
plate may now proceed in a direct fashion 
and is accomplished in a matter of a few 
minutes. The readings recorded are direct 
errors of flatness, and require no calcula¬ 
tions nor use of tie-down points as in the 
usual autocollimator-type calibration. 

A master box-type straightedge 46 
inches in length [1168 mm] is inspected for 
straightness while mounted on the No. 5 
Universal Measuring Machine, Fig. 544. 
The advantage of this type of inspection 
over that where a surface plate is used is 
that the edge being measured is in a neu¬ 
tral direction to any elastic deflection 
which may occur in the straightedge due 
to its own weight. 



FIG. 544 —The unusual capacity of the No. 

5 Universal Measuring Machine is 
demonstrated. Straightness of a fit-inch 
long [1168 nun) master box-type straightedge, 
is being inspected. The edge under 
inspection is in a neutral direction to any 
deflection -which may occur. 
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FIG. 545 —.4 :Ao» Universal Measuring to the front, of the cross-carriage in place of 
Machine has been Modified for sfipecicd the spindtk Special munity elements and 

applications for the Bausch & bomb Optical clutch allow the machine to be traversed 
Company. A mounting plate is fastened either rapidly or in fine increments. 



SPECIAL DESIGNS 

The skills, techniques, masters and gages 
which are employed to make standard ma¬ 
chines are also adaptable to modified or 
special-purpose machines. 

A No. 4 Universal Measuring Machine, 
modified for special application for the 
Bausch & Lomb Optical Company, is 
shown in Fig. 545 undergoing final calibra¬ 
tion by comparison to Step Gages. Mount¬ 
ed on the cross-carriage (no spindle used) 
is a scraped flat plate containing a pattern 
of tapped holes to facilitate the mounting 
of various measuring fixtures. To the right 
in the foreground is a slewing motor at¬ 
tached directly to the lead screw and used 
to transport the machine table rapidly. To 
the left, foreground, is a stepping motor 
which drives the lead screw through 20 to 1 
gearing when fine resolutions of movement 
are needed. The stepping motor is disen¬ 
gaged by a clutch arrangement when the 
slewing motor is used to drive the table. At 
the end of the lead screw is a coarse en¬ 
coder for establishing rough location. 

72-Inch Measuring Machine 

The 72-inch Measuring Machine shown in 
Fig. 546, unlike other Moore Measuring 
Machines, is a single-axis design. Its func¬ 
tion is to calibrate lengths up to 72 inches 
[in metric 1830 mm] swiftly and accurately. 
Cylindrical guide ways are used rather 
than V-ways. The carriage is lightly con¬ 
structed and non-friction rollers are used to 
allow movement with only feather force. 

In place of the standard lead screw nut, 
a single-tooth nut (or wedge) is used to 
contact the thread flank of the lead screw. 
The “nut” may be disengaged from the 
screw, which frees the carriage to be moved 
with great speed by hand. The nut, when 
re-engaged, is done so very accurately with 
a near-perfect double-cone and ball pivot, 
pulling the carriage into position repeat- 
ably to within 1 or 2 millionths of an inch 


fuj, (pin —The 72-inch Single Axis 
Measuring Machine has the capacity to 
measure lead screws, end standards, and 
line standards up to 72 inches in length 
[in metric 1880 mm], A unique disengage- 
able “nut” allows the carriage to be moved 


rapidly along on rollers and cylindrical 
ways to the nearest, .100 inch [in metric 
;.? mm]. Finer increments are established by 
engaging the nut and selling as with a 
standard machine. 











[1/20 Aim or less]. By this means, every 
0.100 inch [3 mm in metric machines] is 
established by using the lead screw as a 
length standard, and without rotating the 
lead screw. Increments within 0.100 inch 
[3 mm in metric machines] are secured by 
re-engaging the nut and rotating the screw, 
displacements being read on the lead screw 
dial. 

The advantage of this design is the vir¬ 
tually instantaneous measurements which 
may be taken of any length within its 
range, without heat built-up in the lead 
screw. The master and the part to be 
checked both rest on the base of the ma¬ 
chine, which acts as a heat stabilizer to 
asssure equalization of their temperatures. 

The 72 inch [in metric 1830 mm] Mea¬ 
suring Machine is presently being em¬ 
ployed to calibrate the longer lead screws 
used in large Measuring Machines in the 
course of thread-grinding and corrective 
lapping. It is also suitable for calibrating 
end standards and line standards. 

Numerical Control of Measuring 
Machines 

The lead screw is ideally suited to numeri¬ 
cal control, since the control system must 
only, in effect, accurately index the lead 
screw by use of resolvers or transducers 
through the correct number of whole turns 
or fractional turns. 

Basically there are two approaches 
which may be considered for numerical 
control of a Measuring Machine. -In the 
first approach, the Measuring Machine is 
commanded to traverse to an intended 
position. The deviation from true position 
is read on the electronic indicator, or may 
be registered on a strip recorder, or even 
printed out. 


In the second approach, the Measuring 
Machine is made to “hunt” for position 
(for example, the location of a hole) with 
a special type of comparator* through a 
servo feedback system, perhaps through a 
computer to the lead screws. The first ap¬ 
proach is relatively simple and has already 
been applied successfully on many Moore 
Jig Borers, Jig Grinders and Measuring 
Machines. The second approach has not 
yet been used, although it has been seri¬ 
ously discussed by some firms. The second 
approach will almost certainly be more 
complex and more costly, although within 
present capabilities. 

The author is inclined to think that the 
first approach is more realistic since it al¬ 
lows the use of standard components, and 
cost will be less. Also, although the second 
approach gives actual coordinates of the 
part, present availability of computers al¬ 
lows such information as obtained in the 
first approach (deviations from nominal) 
to be readily converted to coordinate 
values, at much less investment. 

Numerical Control may be adapted to 
Measuring Machines in so many various 
arrangements and combinations, that the 
particular design arrived at by the user is 
normally decided upon only after engineer¬ 
ing consultation. The two basic control 
systems used, however, are “point-to- 
point”, when the number of positions is 
relatively few (as in hole-location), or 
“continuous path,” when many coordinate 
positions are to be recorded, as in a cam 
or template.** 


*The Sperry Gyroscope Company makes a com¬ 
parator of this type called a “Sperry Powered In¬ 
spection Probe.” See “Industrial Controls and In¬ 
strumentation,” Publication of Sperry Gyroscope 
Company, Ltd., Canada, June, 1966. 

**For more detailed information see: “Moore 
Numerically Controlled Continuous Path Jig Grind¬ 
er with General Electric Mark Century Numerical 
Control, Pamphlet No. GEA-8701, General Electric 
Company, Schenectady, New York, 1969; “Adap¬ 
tation de l’outillage a la commande numerique,” 
Usine Automation, May, 1967. 

See also articles listed in footnote No. 1, page 318. 



5. Ruling Engines— 
Diffraction Gratings 

DIFFRACTION GRATINGS 

“No single tool,” observes George R. Har¬ 
rison, “has contributed more to the prog¬ 
ress of modern physics than the diffraction 
grating, especially in its reflecting form.”* 
Gratings are applied to a wide variety of 
scientific and industrial tasks such as me¬ 
trology (moire fringes), radioastronomy 
(microwave gratings) and optical image¬ 
forming (fresnel lenses). Optical gratings, 
depending on their intended purpose, may 
be of different types, such as concave or 
flat, and may be used either in transmission 
or reflection. George W. Stroke notes in 
“Diffraction Gratings”: “When gratings 
are used for spectroscopic purposes, their 
function is to separate the diffracted wave 
fronts of different frequencies (or wave¬ 
lengths) by angles which are sufficiently 
large to permit easy resolution by an ap¬ 
propriate optical system and receptor.”** 
Spectrographs utilizing diffraction grat¬ 
ings have provided most of the information 
we now have on the structure of atoms and 
molecules. The diffraction grating, it would 
appear, has been aptly termed by Henry 
Norris Russell “the master key of sci¬ 
ence.”*** 

*George R. Harrison, “The Production of Diffrac¬ 
tion Gratings—1. The Development of the Ruling 
Art,” JOSA, June, 1949, p. 413. 

**George W. Stroke, “Diffraction Gratings,” 
Handbuch der Physik, p. 432. 

***George R. Harrison, “The Controlled Ruling of 
Diffraction Gratings,” Proceedings of the American 
Philosphical Society, October, 1958, p. 483. 
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An important category of diffraction 
grating is the flat, reflective type. It con¬ 
sists of a blank of optically-flat quartz, 
glass, pyrex, or other ceramic material such 
as the new composition Cer-Vit.* On the 
functional surface of the blank is a thin 
metallic coating. While Speculum metal 
was used most extensively in the past, 
today a layer of aluminum, deposited by 
vaporization is usually used. Thickness of 
the deposit is no more than a few >im 
[1/10,000 inch or so]. 

As many as 30,000 grooves per inch 
[1181 per mm] are furrowed (not cut) by a 
diamond tool into the aluminum strata. 
The diamond tool, as shown by J. Robert 
Moore, may vary in shape depending on 
the characteristics which are sought.** 
In most gratings, metal is displaced or 
burnished, but not removed. 

Ruling of the grating is performed by 
“ruling engines,” designed and constructed 
to near mechanical perfection. The level of 
this accuracy may be appreciated when one 
considers that for ruling a 10-inch grating at 
300 per millimeter spacing [7620 per inch], 
the ruling engine must traverse the equiv¬ 
alent of 15 miles [24.140 kilometers], and 
yet maintain prescribed spacing and par¬ 
allelism within a small fraction of a light 
fringe.*** A 6-inch grating of 30,000 lines 
per inch [1181 per mm] produced at Jarrel- 
Ash is said to require 12 days of continuous 
ruling.**** 

For a grating to be classified “excellent,” 
it must be accurate to within at least 1 
millionth of an inch [0.000025 mm], accord¬ 
ing to Harrison. Special gratings having up 


*Trade name of a material manufactured by 
Owens-Illinois Inc., Toledo, Ohio. 

**J. Robert Moore, “Precision Diamond Tools for 
Ruling Diffraction Gratings, etc.,” Petersham, 
Massachusetts, August 1, 1968. 

***George W. Stroke, “Diffraction Gratings,” 
Handbuch der Physik, 1967, p. 702. 

****Ibid., pp. 705-706. 


to 270,000 grooves per inch [10630 per 
mm] have been produced—a fact which 
may impress the reader, but not the spec- 
troscopist who is concerned mainly with 
the quality, spacing and parallelism of the 
lines.* As Harrison states: “As many as 
100,000 grooves to the inch [3937 per mm] 
have been ruled on inferior engraving 
engines.”** 

For some period of time it was falsely 
assumed that resolving power of a grating 
could be enhanced only by increasing its 
number of grooves. Spectroscopists now 
acknowledge that the ruled width of the 
grating is the more critical factor in resolu¬ 
tion. The perfection of gratings in greater 
widths is perhaps their greatest goal. 

In order for the engine to perform to its 
full potential, the environment in which it 
operates must receive elaborate attention. 
The operation of an early engine by Blyths- 
wood was reputed to have been ruined by 
the natural movement of trees in the vi¬ 
cinity during a heavy wind; R. W. Wood 
refers to the effect of passing streetcars.*** 
In order to negate these vibration effects, 
the engine is supported on huge concrete 
structures and piers, some floated on 
compressed air. Still further damping of 
vibration is obtained through the use of 
alternate layers of cork, sand and other 
materials. Even with these precautions, 
natural phenomena still dictate design. 
Harrison reports that even in a controlled 
environment, earth tides caused slow drifts 
in diamond position when the diamond 
monorail was at first separately supported 
from his engine. 


*Erwin G. Loewen, “Positioning System Spaces 
Lines to Within 1/10 Micro-Inch,” Control Engi¬ 
neering , May, 1963. 

**George R. Harrison, “The Controlled Ruling of 
Diffraction Gratings,” Prooceedings of the American 
Philosophical Society, October, 1958, p. 483. 

***George R. Harrison, “The Production of Dif¬ 
fraction Gratings —1. The Development of the 
Ruling Art,” JOSA, June, 1949, p. 424. 


Temperature of the air surrounding the 
engine is held as close as 0.01° C [0.018° F]. 
In the Spectroscopy Laboratory at Massa¬ 
chusetts Institute of Technology, however, 
one engine is further temperature-stabi¬ 
lized by immersion in an oil bath, con¬ 
trolled to an order of 0.001° C [0.0018° F]. 

Even so, an excellent grating is produced 
only with scientific perspective, devotion, 
ingenuity and instinct compounded with 
luck. Endorsing this thinking, Harrison 
quotes similar sentiments expressed by 
Michelson, the American scientist who 
wrote in Nature: 

“When the accumulation of difficulties 
seems to be insurmountable, a perfect 
grating is produced, the problem is con¬ 
sidered solved, and the event celebrated 
with much rejoicing only to find the next 
trial a failure . . . one comes to regard 
the machine as having a personality . . . 
requiring humoring, coaxing, cajoling, 
even threatening!”* 

Dr. Harrison cogently positions the dif¬ 
fraction grating as to its significance to 
science: 

“It is difficult to point to another single 
device that has brought more important 
experimental information to every field 
of science than the diffraction grating. 
The physicist, the astronomer, the chem¬ 
ist, the biologist, the metallurgist, all use 
it as a routine tool of unsurpassed ac¬ 
curacy and precision, as a detector of 
atomic species to determine the charac¬ 
teristics of the heavenly bodies and the 
presence of atmospheres in the planets, 


*Ibid ., p. 416. 
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to study the structures of molecules and 
atoms, and to obtain a thousand and one 
items of information without which 
modern science would be greatly handi¬ 
capped. The history [of gratings] is of 
special importance in emphasizing the 
basic role of the experimenter, and par¬ 
ticularly that of the perfector of new in¬ 
struments, the availability of whose prod¬ 
uct is all too often taken for granted by 
other scientists.”* 

A master grating may command a price 
as high as $50,000.00. By carefully con¬ 
trolled processes, replicas are made to such 
perfection as to be almost indistinguish¬ 
able from the original. Indeed, in some 
ways they may even be superior.** 

A plane reflection replica grating, ruled 
over an area of 2.5 by 2.5 inches [63.5 by 
63.5mm], having 91,500 grooves per inch 
[3602 per mm] may sell for $1,200.00.*** 

RULING ENGINES FOR 
DIFFRACTION GRATINGS 

(Historical) 

Light that contains a mixture of wave 
lengths, such as sunlight or starlight, is 
broken into its spectrum of individual col¬ 
ors when passing through a prism (see 
“Standards of Length,” Fig. 163). 

. David Rittenhouse, the American as¬ 
tronomer, noted in 1786 the diffraction 
effects when light from a distant lamp was 
viewed through a silk handkerchief. This 
observation led him to construct what was 
perhaps the first grating by laying hairs 
across two fine-pitch screws. 

By 1821, Joseph von Fraunhofer had 
produced diffraction gratings by ruling gold 


* John Strong, * ‘The Johns Hopkins University and 
Diffraction Gratings,” JOSA, 1960, p. 1152. 

**Erwin G. Loewen, “Positioning System Spaces 
Lines to Within 1/10 Micro-Inch,” Control Engi¬ 
neering, May, 1963. 

***Idem. 


film deposited on glass. One grating thus 
ruled was inch [12.7 mm] wide with 4,000 
grooves. It was not until 1870, however, 
that a grating was made which surpassed 
the prism in resolving power. This grating, 
constructed by L. M. Rutherfurd, a New 
York amateur astronomer, was 2 inches 
wide [50.8 mm] and had 35,000 grooves. 

The goal has always been to produce 
wider gratings, for the larger the grating, 
the higher can its resolving power be. 
George R. Harrison observes: “The greater 
the width of ruled surface on a grating, the 
greater is its power to separate close-lying 
wave lengths which may otherwise be in¬ 
distinguishable.”* To separate properly 
the lines of complex spectra such as those 
emitted by uranium or thorium calls for 
gratings ruled at least 12 inches wide 
[304.8 mm]. 

The wider the grating, however, the 
more complex and exacting must be the 
ruling engine and the total environment in 
which it operates. Early physicists and 
technicians did not have available the 
technologies necessary to construct accu¬ 
rate large gratings. Ingalls describes what 
he calls the “seven demons” of friction, 
wear, warpage of the engine ways, creep (or 
instability), vibration, dust, and changing 
temperature which conspire against those 
attempting to rule gratings only a few 
inches [75 mm or more] in width.** In fact, 
an Australian, H. J. Grayson, (circa 1893) 
was so consumed with the ruling engine 
“fever” in life, that his widow burned all 
his papers relating to ruling engines on his 
death. 

Scientists from all over the world con¬ 
tributed to the advancement of the ruling 


*George R. Harrison, “The Challenge of the 
Ruled Grating,” Physics Today , September, 1950. 

**Albert G. Ingalls, “Ruling Engine,” Scientific 
American, June, 1952, p. 47. 


art. In England, for example, a ruling en¬ 
gine was constructed by Lord Blythswood 
in 1880, and was the only source of British- 
made gratings until 1950. The machine, 
which employed a 2 foot [610 mm] diam¬ 
eter, 720 tooth wheel attached to the lead 
screw, was used at the N.P.L. Later it was 
used by Hilger-Watts, Ltd. (now a Divi¬ 
sion of Rank Industries) and is today at 
the Science Museum at Kensington. 

Henry A. Rowland 

—Johns Hopkins University, 1882 

For decades, attempts to rule larger grat¬ 
ings were unsuccessful. Not until 1882, 
when the first of three ruling engines was 
constructed at Johns Hopkins University 
by Henry A. Rowland, a physicist, were 
high quality gratings up to 6 inches 
[152.4 mm] in width generated. Rowland’s 
success was to a great measure attributable 
to improved mechanical design, especially 
to his perfection of lead screws through 
laborious lapping. 

John A. Anderson 

—Johns Hopkins University, 1901 

In 1901, on Rowland’s death, his en¬ 
gines were rebuilt and improved twofold 
by John A. Anderson of Johns Hopkins. 
Anderson’s greatest improvement was in 
the perfection and alignment of Rowland’s 
already superb lead screws. For many 
years, the most precise gratings were pro¬ 
duced by Rowland, Anderson and then 
Remsen W. Wood at this university. 

John A. Anderson, Harold D. 
Babcock, and Harold W. Babcock 
—Mount Wilson Observatory, 1916 

John A. Anderson, on leaving Johns Hop¬ 
kins in 1916, went to the Mount Wilson 
Observatory at Pasadena to work on a 
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huge Rowland-type of design, already sev¬ 
eral years in construction. This machine 
was intended to rule gratings up to 18 
inches by 24 inches [457 x 610 mm]. After 
30 years of effort on this engine, it was 
abandoned. A 10-inch [254 mm] engine 
undertaken by H. D. Babcock in 1930, and 
at a later date continued by son Harold 
W. Babcock produced some excellent grat¬ 
ings up to 7.5 inches [190.5 mm] in width. 
See Fig. 547. 

Albert A. Michelson, Early 1900’s 

A ruling engine was constructed by A. A. 
Michelson at the University of Chicago in 
the early 1900’s. Though he attempted to 
produce gratings up to 14 inches [355.6 mm] 
wide, he was largely unsuccessful. A second 
engine was built by Michelson, however, 
and he set his sights a little lower, striving 
instead for 12 inch [304.8 mm] gratings. A 
ball-bearing carriage was employed to re¬ 
duce frictional movement. 

The idea of interferometric control of 
ruling engines can be traced to Michelson, 
although he limited its application to im¬ 
prove his screws, or to test the engine prior 
to ruling. A 9.4 inch [238.76 mm] grating 
was produced with the second engine, the 
largest to that date, but not of a quality 
to match those made on the Rowland 
engines. The second engine, started around 
1910, was still not functioning to expecta¬ 
tion when taken over and completely re¬ 
built in 1930 by Henry G. Gale, another 
physicist at the University of Chicago. 
Gratings produced on this engine still did 
not match the quality of those produced on 
the Rowland engines. 

In 1947 the Michelson-Gale engine was 
given to the Bausch & Lomb Optical Com¬ 
pany, Rochester, New York where it was 
again rebuilt, on this occasion by David 
Richardson and Robert S. Wiley. This en¬ 
gine has since been operated successfully at 
Bausch & Lomb, and has helped to make 
this company the world’s prime supplier 







FIG. 548~~The success of Howland's ruling 
engine is attributable, to his perfection of 
mechanical design and to his attention to 
the construction, mounting and lapping of 
the lead screw. This engine resembles a 
shaper in operation. 



of commercial gratings up to 8 inches 
[203.2 mm] in width. 

The principle by which the Rowland and 
Michelson-Gale engine operated was to 
support the grating blank on the engine 
carriage of a double-V design. The carriage 
was displaced by a single screw, the dia¬ 
mond point being traversed over the grat¬ 
ing at each position. 

John Strong 

—Johns Hopkins University, 1950 

A major contribution to diffraction grat¬ 
ings and ruling engine design was made by 
John Strong of Johns Hopkins University 
in 1950. Strong showed great mechanical 
insight and ingenuity in its construction 
and in aligning and lapping components 
such as the guideways and the lead screws. 
He was especially successful in tackling the 
problem of “Rowland ghosts,” spurious 
spectral lines usually caused by periodic 
error attributable to the screw or its 
mounting. Strong followed Rowland’s prin¬ 
ciple of design as he expressed it in an arti¬ 
cle for Encyclopedia Brittanica entitled 
“The Screw” (Vol. XI): “. . . no workman¬ 
ship is perfect; the design must make up 
for its imperfections.” * This led him to his 
own principle of “. . . overconstraint and 
elastic averaging —a principle which con¬ 
trasts with that of kinematic design—in¬ 
struments of the very highest order are 
not attainable after the teachings of the 
latter.”** Strong seemed to reinforce the 
philosophy of this text when he stated: 
“I find that the construction methods of 
greatest precision are all primitive meth¬ 
ods.”*** As examples, he cites Whitworth’s 
three-plate method and the generation of 
a 360 tooth dividing head to index the 


*John Strong, “New Johns Hopkins Ruling En¬ 
gine,” JOSA, January, 1951, p. 5. 

p. 7. 


screw with “unmeasurable errors” (a prin¬ 
ciple not unlike that of the 1440 Index; 
see pages 233-235). 

The Rowland-type of engine resembled 
a shaper (Figs. 548, 549), whereas the de¬ 
sign showed by Strong in 1951 resembled 
a planer (Figs. 550, 551). The principle of 
operation of both the Rowland and Michel¬ 
son-Gale engines was to support the grating 
blank on the engine carriage as it was dis¬ 
placed by a single screw while the diamond 
point traversed the grating at each posi¬ 
tion. In the Strong engine, the diamond 
carriage is advanced instead by two op¬ 
positely placed, counter-rotating screws. 
The purpose of this design is to nullify the 
effect of heat on the screws, since they ex¬ 
pand in opposite directions. The intercon¬ 


nection of the nuts is used to carry the 
ruling diamond. In marked contrast to the 
Rowland design, the grating blank itself is 
reciprocated, controlled by the guide-ways 
under the diamond carriage after the latter 
is stopped. 

This design removes the weight of the 
grating blank from the displacing member, 
and makes “fan errors” of the grating due 
to way curvature impossible, although the 
lines still may not be straight. 

At first, gratings were scribed on hard 
speculum mirrors which deteriorated the 
diamond rapidly. Strong made an impor¬ 
tant contribution to the grating itself by 
introducing the still accepted process of 
vaporizing a thin layer of soft aluminum 
on the blank to be ruled. 





FIG. frio — H. A. Howland ran be credited 
■with fathering the ruling engine ad for 
diffraction gratings. He is shown with one 
of his ruling engines at Johns Hopkins 
Uv iversity. 

Courtesy Journal of the Optical Society of Amenca. 




George R. Harrison—Massachusetts 
Institute of Technology, 1948. 

The first Michelson engine was sent from 
the University of Chicago to M.I.T. in 
1948, where it was rebuilt by George R. 
Harrison, a spectroscopist (see “Introduc¬ 
tory”). Although O’Donnell of the Uni¬ 
versity of Chicago noted that the buttress 
screw of 2mm [.079 in.] lead of this engine 
was originally corrected to ±0.1 fringe 
periodic error and 0.7 fringe cumulative 
error, Harrison found an increased error 
because the screw was bent—a conse¬ 
quence of prolonged support on its ends 
while the machine lay idle during the years 
of World War II. 

The method introduced by Harrison was 
to revolutionize the art of controlling rul¬ 
ing engines. He used a Michelson-type in¬ 
terferometer and the green line of an iso¬ 
tope of mercury (Hg 198) as a light source 
to measure carriage displacement. Through 
a servo feedback, the engine was controlled 
continuously. This “Commensurator,” as 
it was called, enabled improvements in 
gratings beyond the pure mechanical pre¬ 
cision of the lead screw and the engine, 
Figs. 552 and 553. In addition, Harrison 
recognized early that variations in atmos¬ 
pheric pressure could alter the wavelength; 
he therefore compensated for this natural 
variable in his design. 

By 1949, Harrison had made appreciable 
progress with this technique, but reported 
that 8-inch [203.2 mm] gratings with a 
resolving power of 400,000 were not yet 
within the capability of ruling engines.* 

In 1955, Harrison and Stroke reported 
significant advances with the ruling engine 
at M.I.T. through interferometric control 
and continuous carriage advance. In this 
technique, the blank moves forward at a 
constant speed while the diamond rules 

*George R. Harrison, “The Production of Diffrac¬ 
tion Gratings—1. The Development of the Ruling 
Art,” JOSA, June, 1949, p. 416. 


fig. 550 . -John Strong followed the 

principle of Henry Howland that design 
must compensate for mechanical 
imperfection. To this principle he added 
Ms own.precept of “elastic averaging,” 


The result was a riding engine that 
differed mbsinnlially from that of 
Rowland, resembling a planer in design. 
Both the Rowland and Strong engines, 

however, achieved success. 
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pig;, foi- John Strong of Johns Hopkins 
Liniversifg fsade many important 
amlnhutions to i tiling engine design rend 
diffraction gratings. The success of h is 
engines can he attributed to mechanical 


insight and. devotion to accwaci/ He is 
shown with W. 11. Percy (left) looking a 
the riding engine he designed while at 
J alms Hopkins, 

Cour ic-sy jo h ns Ho pk ins Univorsity. 


the grooves. They reported excellent grat¬ 
ings of 8-inch [203.2 mm] ruled width and 
5-inch [127 mm] groove length. In addition, 
problems relating to inertia and stopping 
“stiction” were virtually eliminated, with 
no apparent deleterious effect on the grat¬ 
ing.* 

Interferometric control, it should be 
noted, did not rule out the need for me¬ 
chanical perfection, but extended its limits. 
The choice still had to be made as to 
whether servo-control or mechanical means 
were best to apply corrections. Interferom¬ 
etry and servo-control could more easily 
correct for overall groove spacing than for 
rapid errors. 

By 1959 Harrison and his associates at 
the M.I.T.’s Spectroscopy Laboratory had 
ruled more than 100 test plane gratings 
by interferometric control, seven of these 
in excess of 10 inches in width [254 mm]. 
Harrison comments on these gratings: 
“. . . these [specimens] show the weakest 
Rowland ghost and satellite intensities 
yet reported for ruled gratings.”** Peri¬ 
odic error of the screw was reduced to less 
than l/10th fringe. The slow progress that 
has been made in making grating blanks 
more readily available is reflected in 
Harrison’s report when he notes that of 
seven 10-inch [254 mm] blanks, two of the 
poorly ruled specimens had to be stripped 
to continue the tests. Recent reports indi¬ 
cate that this problem is being solved in 
various laboratories.*** 


♦George R. Harrison and George W. Stroke, “In¬ 
terferometric Control of Grating Ruling with Con¬ 
tinuous Carriage Advance,” JOSA, February, 1955, 
p. 113. 

♦♦George R. Harrison et al., “Interferometrieally 
Controlled Ruling of 10-Inch Diffraction Gratings,” 
JOSA, March, 1959, p. 205. 

♦♦♦George R. Harrison and Stephen W. Thompson, 
“Large Diffraction Gratings Ruled on a Commercial 
Measuring Machine Controlled Interferometri- 
cally,” JOSA, Vol. 60, No. 5, 1970. p. 4. 



FTG. 552 —The “Commcnsurator” of George 
Harrison of Massachusetts Institute of 
Technology represents■ an important 
historical development, since it was the 
first successful application of 
interferometric control of ruling engines. 

Courlesy George Hamsun, M i T. 
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FIG. 553— George Harrison (left) with 
Stephen W. Thompson at the M.I.T. 
Spectroscopy Laboratory, 1965. The “A” 
engine shown teas originally built by 


Michelson, and was completely re-built and 
adapted to interferometric control at M.I.T. 
by Harrison. 

Courtesy George Harrison, M.I.T. 



In the years since, most of the outstand¬ 
ing ruling engines have been adapted to 
interferometric control, as at Jarrell-Ash 
by Stroke and associates* and also at 
Bausch & Lomb and the Mount Wilson 
Observatory. Interferometric control has 
in each case improved the performance in 
comparison with that attained by purely 
mechanical means. 

The Ruling Engine and 
the Lead Screw 

There is an apparent disagreement as to 
the importance of the lead screw. Bausch & 
Lomb stated in 1957: “. . . the precision 
screw ... is the heart of any ruling en¬ 
gine. . . .”** From his study of existing 
systems up to 1952, Ingalls remarks: “If 
the lead screw did not exist, it would be 
necessary to invent it for the ruling 
engine.”*** On the same topic, Harrison 
stated in 1949: “A popular misconception 
exists that the critical portion of a ruling 
engine is its screw; a screw that will pro¬ 
duce translations precise to one micro-inch 
[l/40th Mm] without need of correcting 
mechanism can be made readily by the 
lapping method of Rowland.”**** He goes 
on to say that Michelson, perhaps to em¬ 
phasize its importance, claimed 15 years’ 
time to produce his screw. O’Donnell, on 
the other hand, observes that the 20-inch 
[508 mm] screw for the M.I.T. engine 
required only 3 months working time 
spread over 1 year. Harrison concluded in 


*George W. Stroke, “Diffraction Gratings,” 
Handbuch der Physik, p. 703. 

**Ibid., p. 706. 

***Albert G. Ingalls, “Ruling Engines,” Scientific 
American. June 1952, p. 51. 

****G eor ge R. Harrison, “Production of Diffrac¬ 
tion Gratings—1. The Development of the Ruling 
Art,” JOSA, June, 1949, p. 418. 
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the same paper, nevertheless, that there 
was no more superior element for displace¬ 
ment than the screw.* 

Stroke mentions . . the successful sub¬ 
stitution of a hydraulic drive for the screw 
in an interferometric velocity of light ex¬ 
periment. . . ,”** and W. R. Horsfield of 
Bermuda has built a successful controlled 
ruling engine with hydraulic blank advance. 

The apparent disagreement as to the 
relative importance of the lead screw is 
probably one of emphasis rather than sub¬ 
stance. The most sophisticated ruling en¬ 
gines will undoubtedly encompass some 
form of interferometric-control in their 
designs. Yet almost every successful ruling 
engine in the past and those under con¬ 
struction or on the drawing board now 
rely on a lead screw for basic translation 
of carriage movement. Careful attention is 
given the lead screw’s design and accuracy, 
and greater perfection is continually being 
sought. 

Even if the engine is interferometrically 
controlled, the lead screw must respond 
sensitively to command and permit ex¬ 
tremely fine displacements of the carriage. 
The more perfect the mechanics, the less 
compensation on the part of the inter¬ 
ferometer, leading to ever-higher accuracy. 

Modern ruling engines for diffraction 
gratings must be a synthesis of the ultimate 
in mechanics and the ultimate in optics 
and electronics. 

M.I.T. “B” Engine with the No. 3 
Measuring Machine Base 

In the early 1960’s, the potential of the 
No. 3 Measuring Machine base design for 
use as a ruling engine became apparent 


*Ibid., p. 419. 

**George W. Stroke, “Diffraction Gratings,” 
Handbuch der Physik, p. 671. 


to Harrison of M.I.T. Accordingly, ar¬ 
rangements were made to modify a base for 
this express purpose. To the base was 
added a monorail diamond carriage as 
designed previously by Harrison for the 
rebuilt Michelson engine. Oil-film ways 
were employed, rather than roller-ways, 
since the slight “flutter” caused by the 
rollers was objectionable. The machine 
was transported to M.I.T. where it was 
adapted to interferometric control by 
Harrison and his associates in the Spec¬ 
troscopy Laboratory. Construction of this 
engine had been financed by the Bausch & 
Lomb Company, and the engine was 
considered to be loaned to M.I.T. by that 
company. Corwin Brumley, Erwin Loewen 
and Robert S. Wiley of Bausch & Lomb 
gave technical assistance, and supplied 
blanks, coatings, ruling diamonds and 
grating replicas for testing. Much experi¬ 
mentation took place with this engine while 
the first M.I.T. engine was ruling gratings. 
By September 4,1963, the first 5 by 10 inch 
[127 by 254 mm] high quality grating was 
produced on this engine. 

At a later date, graphitar* was substi¬ 
tuted for the more usual ball construction 
in the lead screw thrust bearing. Since the 
engine was to operate under continuous 
carriage advance, the slight added “stic- 
tion” was considered less deleterious to 
performance than the slight “noise” of the 
balls, and accidentally brinelled thrusts 
were avoided. Since that time this engine 
has produced “. . . many excellent gratings 
of previously unattained dimensions and 
quality.”** 

Several gratings, as large as 8*4 by 16 J4 
inches [209.55 by 412.75 mm] have been 
ruled on this engine. 


*Graphitar is a product of the U.S. Graphite 
Company. 

**George R. Harrison and Stephen W. Thompson, 
“Large Diffraction Gratings Ruled on a Commercial 
Measuring Machine Controlled Interferometri¬ 
cally,” JOSA, Vol. 60, No. 5, 1970, p. 1. 


One particular grating, “B-033”, of 5 
by 10 inches size [127 by 254 mm], having 
316 per mm [8,026.4 per inch] groove spac¬ 
ing, is described by its user as “the best 
ever produced.”* 

The performance of this engine was 
summarized in 1969: “Several dozen grat¬ 
ings in sizes from 125 x 260 mm to 210 x 
410 mm (4.92 x 10.24 inches to 8.27 x 16.14 
inches) have been ruled on the “B” engine, 
during the past three years, of which a 
dozen or more have not only exceeded 
previously available gratings in size, but in 
quality as well.”** 

M.I.T. “C” Engine with the No. 4 
Measuring Machine Base 

Since the early 1960’s, technology has pro¬ 
gressed significantly. Laser interferometry, 
for example, has made possible light beams 
of great strength, maintaining light wave 
stability and coherency over long optical 
paths. 

Use of the Mercury 198 light source at 
M.I.T. was replaced by a “Spectra-Physics 
119 controlled laser” in 1966. When the 
last grating (A-200) was produced on the 
A engine in 1966, the engine was trans¬ 
ported to the Bausch & Lomb Company. 
According to plan the No. 3 base engine 
(“B”) is also slated to go to the Bausch & 
Lomb Company. 

At Moore, improvements were made in 
lead screw accuracy and design. Tech¬ 
niques of roller-way construction were per¬ 
fected. In 1966, a demonstration of re- 


*Unpublished letter to George Harrison from 
Dr. John Evans, Superintendent of Sacramento Peak 
Observatory, December 29, 1969. 

**George R. Harrison and Stephen W. Thompson, 
“Large Diffraction Gratings Ruled on a Commercial 
Measuring Machine Controlled Interferometrically,” 
JOSA, Vol. 60, No. 5, 1970, p. 3. 

The original engine at M.I.T. is designated “A”, 
the modified No. 3 base “B”, and the modified No. 
4 Measuring Machine as “C”. 



peatability of a No. 4 Measuring Machine 
using fiber-optics* was performed for 
Harrison. The repeatability and sensitivity 
of the lead screw using a roller-way design 
shown in this experiment, plus Harrison’s 
further tests of the machine at Moore by 
laser interferometry, convinced him that 
this design was essentially correct for the 
basis of a ruling engine capable of pro¬ 
ducing unusually large diffraction gratings. 
A No. 4 Measuring Machine was then 
modified for introduction of interferometric 
control according to Harrison’s design, 
by Albert E. Johnson, Chief Project En¬ 
gineer at Moore, aided by engineer Vilmars 
Fimbers and various technicians. 

In place of the cross rail member, a 
lapped Nitralloy cylinder held to close 
tolerance of roundness and straightness is 
used as a monorail to guide the diamond ele¬ 
ment. Closer to the grating, the diamond is 
guided by a master Nitralloy straight-edge. 
Shown in Fig. 554 is the master box-type 
straightedge as it is used to establish the 
V-mounting surfaces for the monorail. Fig. 
555 shows the nearly completed machine. 

Mounted on the main carriage is a “yaw 
table,” or compensating plate, used to 
eliminate the approximately J 4 second of 
arc error of carriage curvature of travel. 

Aside from greater size and accuracy, 
this engine has many practical improve¬ 
ments over the previous engines at M.I.T. 
The “A” engine could operate with a 
diamond stroke speed of 2 inches per 
second [50.8 mm/second]; the “B” engine 
(No. 3 Moore base) operated at 3 inches 
per second [76.2 mm/second]. The “C” 
engine (No. 4 Measuring Machine) strokes 
at rates up to 6 inches per second [152.4 
mm/second] without undue vibration. As 
shown, this engine has two sets of mono- 
rails, diamond carriages and diamond 
drives, running in opposite phase and 

*Wayne R. Moore, “Repeatability to 0.2 mil¬ 
lionth!” American Machinist , August 29, 1966. 







fig. 656 - Overall proportion of the 
carriage members is tested against the 
completed base of a ruling engine for 
Professor George W. Stroke of the Sla te. 


University of New York, Stony Brook, 
Wooden models are often used to obtain 
correct configurations prior to actual 
construction. 


spaced 200 mm [7.874 inches] apart to allow 
simultaneous ruling of two blanks. In this 
manner, whether the error is attributable 
to either the interferometer, the carriage, or 
the diamond position can be differentiated. 

George W. Stroke 

George W. Stroke made significant con¬ 
tributions to the ruling engine while at 
M.I.T., and was an early disciple of and 
later a strong advocate of interferometric 
control. After M.I.T., Stroke went to the 
University of Michigan and together with 
O. C. Mohler, F. Denton and T. Harada 
conceived a different design of ruling en¬ 
gine intended to make gratings 32 by 20 
inches [813 x 508 mm]. The blank itself was 
kept stationary; the diamond carriage was 
transported by two lead screws, each in- 
terferometrically controlled. Its base and 
carriage were built at Moore, using stan¬ 
dard equipment, masters and techniques. 

Stroke then transferred the ruling engine 
along with most of his equipment to the 
State University of New York, Stony 
Brook,* where it was decided to complete 
the mechanics of the engine at the Moore 
company. There it is still undergoing con¬ 
struction and design changes. Overall con¬ 
figuration of the diamond carriage element 
is shown being tested using wooden models, 
Fig. 556. The diamond-lifting mechanism 
is shown under assembly in Fig. 557. Its 
function is to feed the diamond gently onto 
the grating (controlled by a ramp), lifting 
it to clear on the return stroke. 


*“New Holography Lab for Stroke,” Scientific 
Research, January 22, 1968. 
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Fig. 558 shows the main elements of the 
ruling engine assembled. The system of 
the ruling engine can be considered to have 
two main structures. In the foreground is 
thefirst structure—the engine proper. Twin 
carriages are used to carry the monorail, 
which guides the diamond carriage and 
diamond-scribing element. Attached to the 
lead screws which displace each table is a 
master worm and gear along with further 
reduction gearing to a stepping motor. 
After scraping and lapping of the carriage 
ways, Rulon is applied to the table to re¬ 
duce friction. 

A reflecting mirror is attached to each 
table as part of a laser-interferometer ar¬ 
rangement. By means of a fringe-counting 
and electronic feedback system, the step¬ 
ping motors attached to each screw are 
pulsed to maintain uniform spacing and 
parallelism of the grooves. 

The monorail consists of mechanical 
elements similar to those in the M.I.T. 
engines. Both tables of the ruling engine 
move continuously while the grating blank 
is being scribed. This procedure is followed 
to avoid errors resulting from starting 
“stiction” and non-uniform oil film as well 
as errors from starting and stopping inertia. 
In effect, the grating is ruled while the mono- 
rail is being displaced; but since the ruling 
stroke is performed thousands of times 
faster than the carriage movement, groove 
inclination is negligible. 

To the rear is the second main structure 
of the engine, containing the driving ele¬ 
ment. Its accuracy is not nearly as critical 
as that of the engine proper. Its function, 
nonetheless, is very important. It also con¬ 
tains a movable carriage which is con¬ 
structed to move synchronously with the 
ruling engine by means of a lead screw, 
worm and gear and timing belt. A unique 
design is employed to drive the diamond 









fig. 558 — The ruling engine for the State 
University of New York, Stony Brook, is 
composed of two structures: (/) The 
ruling engine proper which must accurately 


displace the grating ( foreground); (2) The 
driving element used to traverse the 
diamond across the blank in prescribed 
increments ( background ). 
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element at a constant feed rate across the 
grating—an essential condition to good 
gratings. 

Expectations of the No. 4 Measuring 
Machine (“C”) Engine 

The No. 4 Measuring Machine ruling en¬ 
gine was shipped to M.I.T. on December 7, 
1966. It has since been adapted to interfero¬ 
metric control, and has ruled several ex¬ 
cellent gratings of 12 x 15 inches size 
[304.8 x 381 mm]. Further improvements in 
the performance of this ruling engine are 
projected by Harrison: 

“We are now testing and improving a 
larger controlled ruling engine, the C 
engine, based on a Moore No. 4 Measuring 
Machine with 30 in. [762 mm] travel. 
Blanks up to 450 x 635 x 130 mm [17.7 x 
25 x 5.1 inches], weighing as much as sev¬ 
eral hundred kg (close to 1000 lbs.) can be 
ruled on this engine. Several 300 x 390 mm 
[11.81 x 15.35 inches] gratings produced 
on it show high speed and excellent grating 
quality at angles less than 45°. A further 
fivefold improvement is hoped for to make 
possible the ruling of very large echelles of 
quality equal to those produced on the B 
engine.”* 

Conclusion 

It is fitting that this text should conclude 
with a description of a tool which is per¬ 
haps the most exacting which man has ever 
constructed to date. This tool will increase 
not only our knowledge of the minute 
atomic structure of elements, but when 


*George R. Harrison and Stephen W. Thompson, 
“Large Diffraction Gratings Ruled on a Commer¬ 
cial Measuring Machine Controlled Interferomet- 
rically,” JOSA, Vol. 60, No. 5, 1970. 


mounted on telescopes, will help analyze 
the light spectra of the most distant stars. 
Thus, such mysteries as stellar composition 
and the age of the universe will be explored. 
The information barriers imposed by man’s 
physical limitations of traveling in space 
may then be overcome, Fig. 559. 

The ruling engine—the most reaching 
development of our mechanical and metro¬ 
logical skill—will contribute in a sub¬ 
stantial way to the furthest extension of 
man’s probing. As Ingalls summarized: “If 
the spectrograph is man’s most potent 
tool, the diffraction grating is his highest 
achievement in mechanical precision.”* 

We at the Moore Special Tool Company 
have been pleased to contribute in some 
way to this scientific advancement. Four 
distinctmechanical arts—geometry, length, 
dividing the circle and roundness—have 
been separately perfected, then combined 
to create the mechanical basis of a ruling 
engine. Each one is vital to performance; 
each mechanical art, in turn, derives from 
the foundations of mechanical accuracy. 


*Albert G. Ingalls, “Ruling Engines,” Scientific 
American, June, 1952, p. 45. 
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fig. 559— Rainbow effect demonstrates the 
ability of a diffraction grating to sort oid 
complex and close-lying wave lengths and 
electromagnetic radiation. Much of our 
understanding of the universe, from the 
minutest atomic nucleii to the far reaches 
of the universe is owing to spectrographs, 
the heart of which is the diffraction grating. 
So vital an element is the diff raction grating 
to experimental physics that it has been 
described as the “master key of science.” 
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Bausch & Lomb, Inc., 321, 324, 328, 332-333 
Beale, Oscar J., 152 
Bearing, 24 ff., 42, 46, 65-66, 78 
Bell & Howell Co., 318 
Bellmouth, roundness, 283 
Bending, 21-23, 31 ff., 82, 123-124, 129 
(see Deflection, elastic) 


Bendix Corp., Micrometrical Division, 124 
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Clock manufacture, 149-150, 159-160, 162 
(see Timex watch) 

Coefficient of expansion, 118, 121-122, 166-167, 
186-188 

Colt Patent Fire Arms Manufacturing Co., 148, 
150, 152 

Column, measuring machine, 62, 63, 86, 89 
Colvin, Fred H., 137 
Commensurator (M.I.T.), 330 
Comparative measurement of length 
(see calibration, length) 

Comparator 

Rogers-Bond (Measuring Machine), 153-154 
Stocker & Yale visual, 298 
Compound slide, 54, 61 
(See also cross-slide) 

Concave arc in machine travel, 47-48 
Concentricity, 191, 220, 224, 235, 257, 260-261, 
267-270, 274, 307 
Concord Control, Inc., 318 
Conduction, 118-119 

Conical part, inspection on Universal Measur¬ 
ing Machine, 314 


Connecticut Valley, 147 ff. 

Conservatoire des Arts et Mdtiers, 153 
Continuous path control system, 325 
Continuous Path Jig Grinder, 318 
Control facilities, measuring machine, 286 
Control systems: 

electronic feedback, 336 
interferometric, 330, 335, 338 
numerical, continuous path, 325, point-to- 
point, 319, 325 
servo-control, 330-331 
servo feedback, 325 
Convection, 118-119 
Convex arc in machine travel, 47-48 
Coordinate location, 46-47, 268, 269, 288 ff. 

machine tools, 159 ff., 185 
Coordinate systems special, 317-318 
Cotton Gin, 146 
Cross-slide, 60, 61, 78-80, 82 
(see compound slide) 

(see Squash, check of cross-slide) 

“C” Ruling Engine (M.I.T.), 333-334, 338 
Crystal Palace Exhibition (1851), 148 
Cubit, the term, 101 
Babylonian, 108 
Egyptian short, 99, 101-102 
Greek, 102 
Hebraic, 99 
Roman, 102 
Royal, 101 

Cylinder square, 38, 86, 89 
Cylindricity, 235, 254 


Dalliba, James, 146 

Dannehower, Gilbert L., 155-156, 158 

Darby, Abraham, 139 

Darling, Samuel, 151 

Darwin, 206 

Datum, 34-35, 40-41, 46, 212 
change of during inspection, 28 
for inspecting location, 291-294 
Davies, Charles, 107 
Da Vinci, Leonardo, 103, 138 
Davison, C. S., 143 
Decimal system, 106-107 
Deflection, elastic, 17-18, 27-28, 35-36, 47-48, 
67, 116, 123 
(see Bending) 

Denton, Frank, 335 
Design, machine way, 44-45, 54-56 
Deyrup, Felicia Johnson, 146 
Diameter, 273-275 


Diametrical: 

defining roundness, 274-276 
roundness measurement, 256 
Diamond, 336-337 
probe tip, 131, 133, 273 
for ruling engines, 326, 329, 333-336 
for scales, 114 
Die, increasing life, 285 
Diffraction gratings, 186, 221, 325 ff. 
history of, 327 ff. 
reflective type, 325-326 
telescope, use of, 338 
Digit, 101 

Division of labor, 147 

Dixi, S.A. (Le Locle, Switzerland), 23, 160-161 
jig borer, 161 
Dowel pins, 255 
Drive for ruling engine, 334 
Drive, hydraulic, 333 


Echelle, precision grating, 338 
Edgar, King, 104 

Egyptian system of weights & measures, 98-101 
Elastic averaging, 329-330 
Elastic deformation, 129 ff., 323 
Elasticity, 26 

(See also bending) 

Electrical discharge machining (EDM), 317 
Electronic feedback system, 336 
Electronic indicator, 68 ff. 

Eli Whitney Metrology Laboratory, 136 
Eltee Manufacturing Co., 317 
Emery, J. Kenneth, 112-113 
End measure system, 184-185 
End Standards, 115-117, 121-123, 165-166, 
175, 180-182, 212, 322, 325 
16 inch, 183 
master, 175-176 
Moore, 167 
Enfield Armory, 149 
Engelhard, E., 96, 109, 111, 121, 166 
England, early standards, 104 
Environment, conditions of, 117 
Error: 

angular, 224-226, 315 
center-distance, 224 
fan gratings, 329 
of flatness, 323 
locational, 282, 284 

periodic, 198, 217-218, 224-226, 313, 329, 331 
roundness, 257 

from temperature stratification, 320 
tracing, 260 
worm & gear, 241 
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Eskilstuna Rifle Factory, 154 ff. 
Evans, Dr. John, 333 
Edge-finder, 299-303 


Fabry, C., 95, 109 
Fan errors, gratings, 329 
Farago, Francis T., 164 
Federal Products Corp., 70, 71, 131 
Fiber-optic probe, 320, 322, 334 
Fimbers, Vilmars, 334 
Finney, Burnham, 160 
Fizeau, F., 113 

Flat, optical, 112, 113, 230-231 
Flatness, 14, 22, 29, 88, 125, 136, 166-167, 212, 
259, 262, 269-271 
surface plate, 29-30, 285 
workpiece, 282-283 
Foot, 103 ff. 

Egyptian, 101 
Pied de Roi, 103 
Ford, Henry, 158 
Form, of V-way, 36-38 
France, early standards, 103 
Fresnel lenses, 317, 325 
Friction, effect of, 318 
Fringe: 

formation, 112-113 
interference, 112-113 
patterns for flatness, 112 


Gage: 

air, 264, 266 
bore, 301 

edge-finder type, 299-304 
height, 284 
hole location, 281 
master ring, 264, 266, 307 
surface, 27-28 
twist, 26-28, 76, 78 

Gage block, 11, 38, 115, 121-122, 125, 134-136, 
154-155, 211-213, 246, 272, 284, 302 ff. 
angle, 228-230 
Hoke’s, 158-159 

Johansson’s, 154-155 ff., 228-230, 236 
material of, 121-122 
wringing, 115-116 

Gage pin, 40, 212-213, 255-256, 284, 295-296, 
301, 311 

Gaging, 130, 261 
roundness, 256 ff. 
of standards, 129 ff. 


Gale, Henry G., 328 
Galileo, 104 
Galvanometer, 180 
Gardner, Irvine C., 121, 124 
Gascoigne, William, 143 
Gashing, 222-223 
Generator, artwork, 318 

Geometry, 11, 125, 130,136, 166-167, 199, 268, 
280 

hole,165 

inspection with measuring machine, 270-272 
micro-geometry, 124-125, 130 
relationship to length measurement, 124-125 
tooth, 235 

George L. Lincoln Co., 149 
Germany, early standards, 103-104 
Gib, 44, 54-55 
Glazebrook, Richard T., 156 
Gosling, Robin, 143 
Granite plate, 249 
Graphitar, 333 
Gratings: 
micro wave, 325 
replica, 327, 333 
(see Diffraction Gratings) 

Gravity, 104, 107 
Grayson, H. J., 327 
Grazing lines, 125-126 
Greece, early standards, 102-103 
Grinding, 143, 255, 314 
aspheric lenses, 318 
machine, 235 
techniques, 283 
thread, 189-191, 325 
wheel, 235, 283 
Gun-boring: 
machine, 139 
mill, 140 

Gunsmith, 147-148 
Gyroscope, 285 


Hague, The, 139 
Hallock, William, 104, 106 
Hanson, B. M. W., 161 
Harada, Tatsuo, 335 
Hardness: of gage blocks, 121 
of nitralloy, 188-189 
Harkness, William, 99 

Harrison, George R., 186, 318, 325-327, 330- 
334, 338 

(also see Introduction) 

Hassler, Ferdinand, 107 
Hawkins, Gerald S., 205 
Heat cycle, 173-174 


Hemisphere, inspection of trueness of rotation, 
227, 264 

Hertz, Heinrich, 130 
Hertzian formula, 130-132 
Hildreth, Charles, 150 
Hilgard, J. E., 152 

Hilger & Watts, Division of Rank Industries, 
72, 134, 327 
Hobbing cutter, 223 
Hoke, William E., 158 
Hole location, 263, 280 
by coordinates, 288 ff. 
machine tools, 137 ff., 147 ff., 159 ff. 

Hooke, Robert, 130 

Hooke’s Law, 130 (see Young’s modulus) 
Horsfield, W. R., 333 
Hubbard, Guy, 146-148 
Humidity, 29, 120 
Huntsman, Benjamin, 138 
Huyghens, Christiaan, 104, 109 


Inch, 102 ff., 153 
Canada, 108 
England, 108 
International, 166 
Pratt & Whitney, 152 
United States, 108 
Index Center, 250, 313-314 
Index, 1440, 202, 224, 231-232 
Calibration of, 236-237 
rotary table calibration with, 237-241 
Index plate, index center, 313-315 
Index plate, rotary table, 218-221, 314-315 
Indexing, 218-222, 329 
angular, 313 

automatic, 220-223, 234-235, 239-241 
repetitive-type, 313 
Indicator, 

absolute measurement, 304-305 

mechanical, 68 

(see also Electronic indicator) 

Indicator probe, shapes and uses, 28, 129, 130, 
131, 132, 133, 180, 181, 183, 246, 247, 256, 
258, 262, 273, 274, 294, 295, 296, 320, 322 
Indicator and stand 
used in geometry, 22 fi. 
used with overhead spindle, 260-263 
Indus civilization, 98-99 
Industrial Revolution, 137-140 
Inertial guidance system, 285 
Ingalls, Albert G., 327, 332, 338 
Inscribing circle, roundness, 273-274 
Inspection, requirements of tool, 19 
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Instability: 

metallurgical, 106, 121, 236 
(also see stability, dimensional) 

Institution of Mechanical Engineers, 144 
Instituto Galileo Galilei, Milan, 179 
Interchangeability, 107, 136 ff., 144 ff., 159, 
280 

Le Blanc, 146 

(see American System) 

Interference microscope, 128 
Interferometer: 

angle-measuring 232, 233 
compared to 1440 Index, 243 
fringe counting, 117 
Hilger-gauge, 134 
Michelson-type, 114-115, 233, 330 
Koesters-Zeiss, 109 
laser, 110 ff., 186-187, 333 ff. 
piezoelectric, use of in, 111 
Tsugami gage, 134-135 
Interferometric control, 330-335, 338 
Interferometric measurement of length, 117- 
118, 134 ff. 

Interferometry, 109, 167, 330-331 
laser, 110, 111, 186, 321, 333-334, 336 
Internal comparators, 307 
differential-type, 307 
International Business Machines, 321 
International Conference of Weights & Mea¬ 
sures (1927), 109 

International Convention (1960), 109 
International Metric Convention (1875), 107- 
108 

International Prototype Meter, 107-109, 114— 
115, 117 

International Solar Union (1907), 109 
Invar, 121-122 
jig borer housing, 164 


Jarrell-Ash Co., 326, 332 
Jefferson, Thomas, 106, 108, 146 
Jig Borer, 149, 159 ff., 280 
compound type, 53-54 
horizontal spindle type, 52-53 
planer type, 51-52, 319 
Pratt & Whitney Co., 161-162 
of Dixi, S.A. 160-161 
of SIP, 160-161 

Jig grinder, 162 ff., 235, 259, 280 
continuous path, 318 
Jo Blocks, 158 

(see also gage blocks) 


Johansson, Carl Edvard, 154 ff., 212 
Johansson Co., 154-155, 157 
Johns Hopkins University, 327, 329-331 
Johnson, Albert, 166-167, 239, 241, 334 
Jones & Lamson Machinery Co., 148 


Kendall, Nicanor, 148 
Kettering, Charles F., 156 
Keystone Carbon, 317 
Kilogram, 108 
Kleinsorge, Adolph, 21 
Koesters, W., 109 
Kogurt, P., 136 

krypton 86, 107-111, 117, 136, 165, 182 


Lagrange, 206 

Lapping, 21, 76-80, 90, 115-116, 121, 155-156, 
165-167, 177, 182-183, 190, 191, 223-224, 
233-235, 264-266, 327, 329, 336 
Laps, master, 65-68 

Laser interferometry, 110, 111, 186, 321, 333- 
334, 336 

(see Spectra-physics laser) 

Lathe, 139 
pole, 137 

screw-cutting, 137-138, 142 
slide rest, 137, 142 
Vaucanson, 139 

Lathe, diamond-tool facing, 318 
Lauerman, A. L., 130 

Lawrence Radiation Laboratory, U. of Califor¬ 
nia, 270-272, 318 
Lead checker, 190-191 

Lead screw, 47, 224, 290-291, 299, 301, 303-304, 
306, 309, 319-320, 322, 324, 327, 329-330, 
333-336 

assembly diagram, 187 
calibration, 195-198 
heat, 185-186 
indexing, 204 
lapping of, 191 ff. 
lineup, 193-194 
as a length standard, 325 
material of, 187-188 
Michelson’s, 330 
Moore, 187 ff. 
nut, 191-192 

oil temperature control in grinding, 191, 224, 
227 

periodic error, 190-191, 196-199 
pitch, 197, 224 
in Ruling Engine, 332-333 
wear, 185 


Lean, 269 
of V-way, 38-39 
Le Blanc, 146 
Length Standards, see: 
cadmium 
cubit 

end standards 

end standards, Moore 

krypton 86 

eight waves 

laser interferometer 

line standards 

Mercury 198 

meter 

Metre des Archives 
pendulum 
Pied de Roi 
scale precision 
Step Gage 

Toise du Grand Chotelet 
yard 

Lenses, aspheric, 318 
Level comparator (N.P.L.), 115 
Levels, electronic, 204 
Light, thallium, 128 
Light waves: 

absolute standard, 108-113 
calibration, 182 
measurement, 186-187 
monochromatic light, 109, 112-113, 232 
relation of physical standards, 116-117 
separation of, 111, 339, (see also diffraction 
gratings) 

theory of, 112-113 
Lincoln Miller, 149, 152 
Line standards, 108, 114 ff., 121, 166, 322, 
Linear Dividing Machine, 150-151 
Linear recorder (orstrip), 126, 190,259, 316-317 
Lobing, 255 ff. 

Location, 159-164, 254, 268-269, 285 
coordinate, 46-47, 268, 288 ff. 
hole, 263, 280 

roundness, in relation to, 269, 284, 285 
Loewen, Erwin G., 125, 129-130, 333 
London, Tower of, 149 
Lord Chancellor Micrometer, 143 
Louvre, Paris, France, 99 
Lowell Machine Shop, 150 
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Machine & Pointer, 160 
Machine tools, 137 ff147 ff. 
coordinate locating, 159 ff., 185 
design 46 ff., 138, 142, 185 
development in England, 142 ff. 
development of standard types, 149 ff. 
Machine ways: 
designs, 54-57, 142 

Moore, 56 ff., 200, 308-309,318-320,324-325, 
333, 334-335 
straightness, 204 
wear of, 48 

Magnetostrictive mounts, 111 
Magnification, 116, 259-260, 296-299, 304, 316, 
320 

Magnitudes: 

angular, description of, 205 ff. 

Mass production, benefit of accuracy to, 285 
Massachusetts Institute of Technology (M.I.T.), 
186, 330-336, 338 
“A” Ruling Engine, 334 
“B” Ruling Engine, 333-334 
“C” Ruling Engine, 333-334, 338 
Spectroscopy Laboratory, 326, 332 
Master: 

female double V, 35-46 
lap, 65-68 

scraping, 59 ff., 65-68 
Material, effect of deformation on, 130-131 
Material of standard, 121 ff. 

Maudslay, Henry, 137, 142-144 
Mean circle, defining roundness, 274, 276 
Measurement: 
absolute, 304-305, 310 
angular relationships, 204 ff., 216 ff. 
of ball bearing race, 273-274, 309 
cam shapes, 310 
comparative, of length, 115 ff. 
of female radii, 299 ff. 
length, with cadmium light, 109, 113 
length, with interferometer, 134 ff. 
length, with monochromatic light, 112 
light waves, 113 ff. 
of male radii, 299 ff. 
with microscope, 296 
“natural”, 98 ff., 104 

orientation of workpiece for, 270-271, 288 ff. 
repetitive type instruments, 307 
by polar coordinates, 294-295 
in science & industry, 136 ff. 
stepless, 185 


Measuring Machine: 

Brown & Sharpe Manufacturing Co., 152 

coordinate (CMM), 164, 280, 298 

column, 62, 86, 89 

control facilities, 286 

hole geometry, inspection of, 270-272 

Moore length-measuring element, 185-187 

quill, 306 

housing, 50, 62-64, 264-266, 270, 305 
spindle housing, 164, 182 
table, 62, 80 ff., 86 

Universal Measuring Machine, 11, 40 
photo of main members, 58 
certificate of accuracy, 281 
control facilities, 286 
cubic concept, 10-11 
techniques & applications, 280 ff. 
Whitworth’s, 144 
yoke, 264-265 

Rogers-Bond comparator, 153-154 
of Maudslay’s, 143 
Measuring force, 129-133 
Measuring Machine Laboratory, 168 ff. 
design features, 173 
temperature-controlled floor, 173-174 
Mendenhall Order, 108 
Mercury 198, 109, 333 
Meter, 104, 108, 109, 152, 153 
Metre des Archives, 104-105, 107-108, 114, 117 
Metric System, 104, 106-107 
Metropolitan Museum of Art, New York, 100 
Meyerson, Melvin R., 121 
Michelson, Albert A., 95, 109, 113, 326, 328- 
329, 332-333 

Michigan, University of, 335 
Micro-circuit analysis, 317 
Micro-densitometer, 318 
Micro-line table, 214-215, 259 
Sine, 314-315 
(see also Sine Table) 

Micro wave gratings, 325 
Micrometer, 73, 151, 185, 255, 258, 301 
Brown & Sharpe Manufacturing Co., 155 
millionth, 246-247 
Palmer, L., 151 
screw, 73, 143, 244-246 
Microscope, 216-217, 296-298, 322 
interference, 128 
optical, 116-117 
photo-electric, 111, 114-117 
Microtecnic Exhibition, Zurich, 263 
Mille (mile), 102 
Millieme (mill), 206 


Mirror: 

housing, 237 

in use with autocollimation, 28, 29, 44, 72,73, 
74, 80, 81, 82, 230, 231, 232, 236, 237, 239, 
240, 241, 248, 249 
Mohenjodaro (stone weights), 98 
Mohler, O. C., 335 
Moird fringe, 185, 216, 325 
(see also Scale, precision) 

Moisture, effect on surface plate flatness, 29-30 
Manochromatic light in length measurement, 
112 

Monorail, for ruling engines, 333-334, 336 
Moore, J. Robert, 140-141, 149, 153, 159, 162, 
326 

Moore, Richard F., 11, 124, 162, 232-233, 295, 
299 

Moore Special Tool Co., 162 ff., 287, 334-335, 
338 

Jig Borer, 162-163 
Jig Grinder, 162-163 
Standards Room, 175 ff. 

Tool Room Division, 285 
Universal Measuring Machine, 162 ff. 
Moore, Wayne R., 320, 334 
Mount Wilson Observatory, 327-328, 332 
Mounting of workpieces, 287 ff. 

Murray, Matthew, 139, 142 
Round Foundry, 139 


Nasmyth, James, 141-143 
National Bureau of Standards (U.S.A.), 11, 97, 
111, 123-124, 156-159, 166-167, 175, 212, 
237 

reports on calibrations, 238 
National Cash Register Co., 162 
National Physical Laboratory (England), 11, 
73-74, 95, 109, 111, 114-116, 131, 156, 
166-167, 175, 228-230, 237, 246, 327 
Naugatuck Valley, 162 
Netherlands Gun Foundry, 139 
New Haven Historical Society, 147 
Newcomen, steam engine, 137 
Newcomen, Thomas, 137, 139, 141 
Nickel, 121-122 
Nickols, L. W., 17, 95, 131 
Nicolau, 125-126 

Nitralloy, 29, 87, 121, 163, 166, 187-188, 334 
hardness, 188-189 
straightedge, 83 
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North, Simeon, 146 

Numerical control, 319, 325 

Nut, Lead Screw, 187, 191-194, 200, 324-325 


Oakley, T. R. J„ 29, 131 
Obelisks, 100 

O’Donnell, T. J., 109, 233, 330, 332 
Oil film, 132 

on machine ways, 318, 320, 333 
spindle, 226-227, 259, 265 
Oil, temperature control: 
lead screw, 200 
thread grinding, 188, 189, 191 
optical systems, 109, 110, 113, 117, 134-136 
orientation of workpiece for measurement, by 
coordinates, 270-271, 288-299 
Oscilloscope (fiber optics), 320, 322 
Out-of-parallel, 282-283 

Out-of-roundness, 255-256, 258, 261, 263, 268, 
273-275, 283 
dowel pins, 255 
nature of, 255-256 
sources, 255 

Out-of-squareness, 34, 48-50 
Out-of-straightness, 257 


Palmcrantz, Hilge, 155 
Palmer, L., 151 
Parallelism, horizontal, 42-44 
Parallelism, vertical, 44-46, 81 
(see Twist) 

Paris Exhibition, 151 
Pendulum, 104, 106, 108 
Penetration, 134, 136 

(see also Phase correction) 

Periodic error, 198, 217-218, 224, 313, 329, 331 
Perkin-Elmer Corp., 110 
Perot, A., 95, 109 
Perry, John, 99, 107 
Perry, Wilbur H., 331 
Phase correction, total, 134, 136 
(see also Penetration) 

Phoenix Iron Works, 152 
Photo-electric microscope, 111, 114-117 
Photocell for fringe counting, 116-117 
Physikalish Technische Bundesanstalt (Ger¬ 
many), 11, 96, 109, 166-167, 237 
Picard, Jean, 104 

Pick-up block, 272, 274, 301-302, 303 ff. 

Pistol, Simeon North, 146 
Pitch: 

lead screw, 197, 224 


Plane: 

creation of, 21-29 
flat (or true), 38, 56, 143 
Plumier, Charles, 138 
Point-to-point control system, 325 
Polar chart, 270 
Polar coordinate, 294-295 
Polar recorder, 259, 264, 268, 270, 271, 278, 
279, 280, 283 
Polhem, Christopher, 138 
Polhem, sticks, 154 ff. 

Polygon, 230-233 
pouce, 103 
Pratt, Francis, 152 

Pratt & Whitney Co., 149, 151-154, 158, 161 
jig borer, 161-162 
rifles, 152, 155 
Prism, optical, 111, 250, 327 
Probe, elastic deformation, 129 ff. 

Probe tip, 273, 295-296, 303, 305 
diamond, 133 

(see also indicator probe shapes and uses) 
Proficorder, Bendix Corporation, Micrometrical 
Division, 126 
Pyramids, 100 


Quill, 306, 316 

Quill, vertical movement, 50, 63, 90, 267, 269, 
270, 296, 306, 307, 314, 316 
Quill housing, measuring machine, 50, 62-64, 
264-266, 270, 305 


Radial, in defining roundness, 259, 274-276 
Radian, 206 
Radiation, 118-119 
Radius, for roundness, 273 

measurement on universal measuring machine, 
299 ff. 

Ramsden, Jesse, 142 

Rank Precision Industries, Ltd., 72, 126, 134 
Readout, 309-310 
comparator-type, 298 
Vu-point, 309 
Reason, R. E., 126, 274 
Rectangular surface plate, 21 
Refractive index, 136 
Refrigeration cycle, 172-174 


Remex Electronics, Division of Ex-Cell-0 
Corp., 309 

Remington, Eliphalet, 148 
rifle, 148, 155 

Renaissance period, 137-138 
Repeatability, 74, 165, 185, 217-218, 298, 318, 
320, 324-325 

Reverse air flow system, temperature control, 
171-173, 176-179 
Revolver, Colt, 148 
Ribbing, 21-23, 26 

of surface plates, 22-23 
Richardson, David, 328 
Rifle: Eskilstuna Rifle Factory, 154-156 
German Mauser, 155-156 
Remington, 148, 155 
(See also Revolver, Colt) 

Robbins & Lawrence, 148-149 
Pratt & Whitney, 152-153, 155 
Ring gage, measurement of roundness, geome¬ 
try and size, 305 ff. 

Riser block, compound type jig borer, 319-320 
Rittenhouse, David, 327 
Robbins & Lawrence Co., 148-149 
Rod, 103 

Roe, Joseph W., 142-143, 146, 154, 162 
Rogers-Bond Comparator, 153-154 
Rogers, William A., 153 
Roller-checking, 133 
Roller ways, 318-320, 324, 333 
Rolt, Frederick H., 128 
Rome, early standards, 102-103 
Root, Elisha K., 149 

Rotary Table, 177, 216-228, 231, 310-311 
calibration with 1440 Index, 237-241 
construction of, 221-222 
used flat, 311 
used horizontally, 311-313 
interferometric calibration, 232-233 
master template, inspection, 227-228 
mechanical cam-compensated, 217-218 
plain-bearing spindle, 226-227 
optical, 216-217 

spindles of, 221, 223, 224, 225, 226, 227 
tailstock unit, used with, 227, 228, 313 
techniques, 294-295 
tilting type, 248 

Universal Measuring Machine, 269 
worm & gear, 218-221, 320, 336 
Rotation, trueness of, 224-227, 244, 259, 263, 
264 

Rouge, for scraping, 24, 28, 66 
Round Foundry, 139 
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Roundness, 11, 224-225, 227, 235, 254 ff., 264, 
268-270, 274, 276, 280, 334 
alignment, 260, 294-295 
bellmouth, 283 
calibration of, 259 
on centers, 294-295 
circumscribing circle, 273-274 
concentric circles, 273-274 
definitions compared, 273-276 
gaging, 256 ff. 
inscribing circle, 273 
location, in relation to, 285 
lobing, 255 ff. 
mean circle, 274-276 
polar recording, 307 
radial, in defining, 274-276 
surface finish, relation to, 272-273 
United States Standard of, 276 
zonal method of defining, 273-276 
Roundness, measuring methods, 256 ff. 
accurate spindle method, 259-260 
circumferential confining gage, 257 
diametrical, 257 

production vs. absolute methods, 258 
with ring gage, 305 ff. 
rotating on centers, 257 
three-point probe, 258-259 
Universal Measuring Machine, 263 ff., 305 ff. 
V-block, 257-258 
Rowland ghosts, 329, 331 
Rowland, Henry A., 327, 330, 332 
Royal Institution, 155 

Rubbing, for bearing when scraping, 24 ff., 35- 
36 

Ruling Engine, 186, 318, 321, 325 ff. 
diamond, 326, 329 

interferometric control of, 328, 330 ff. 
lead screw, 332-333 
monorail for, 333-334, 336 
Rulon, 336 

Russell, Henry Norris, 325 
Rutherfurd, L. M., 327 


Sacramento Peak Observatory, 333 
Sandvik Steel Co., 155 
Sarton, George, 99 
Scale, engraving, 114-115 
Scale, precision, 114-117,122-123,165,185, 323 
circular, 216-217 
Moird-fringe type, 185 
Scamuzzi, Ernesto, 101 
Schmalz, 125-126, 128 


Science Museum, Kensington, England, 98, 
102-103, 105-106, 137-138, 140, 142-144, 
150-151, 327 

Scraping, 143-144, 171, 193, 336 
master, 59 ff., 65-68 
rouge for, 24, 28, 66 
technique, 20, 24-29, 33-35, 65-68 
lead screw nut, 193-195 
Whitworth, 143-144 
Screw: 

fine pitch for grating, 327 
micrometer, 73, 244-247 
Screw thread of wood, 137 
Serrated-tooth, circle divider, 233 ff. 
Servo-control, 330-331 
Servo feedback control system, 325 
Sexagesimal system, 102, 204-206, 219 
Babylonians, 101-102, 205 
Sine bars, 204, 211-213 
Sine block, 211 
Sine plate, 211, 213, 214 
Sine table, 215, 314, 315 
Sine principle, 204-205, 208 ff. 
angular measuring devices, 211 ff. 
application of, 211 ff. 

Sine table, 211 ff., 259 
boxing of, 214-215 
double sine design, 213-214 
inspecting angular relationships, 214 
setting, 214 

applications of, 215. 259. 314-315 
Size, Measuring, 154-155, 254, 264, 268, 276, 
284, 285, 299 ff. 
absolute, 269 

measuring radius, 299-304 
relative, 269 

Small Angle Divider, 231, 243 ff., 280, 315-317 
applications of, 248, 315-317 
calibration, method of, 246-248 
design, 244-245 
1440 positions, 315 
setting an angle with, 245 
Socidtd Genevoise d’Instruments Physique 
(S.I.P.), 160-161 
Sola, Marcos C., 121 
Space program, 285 
Spacing: 

angular, 295 ff., 310-311, 313, 315 
tooth, 235, 311 
Spangberg, Arthur, 156 
Spectra-Physics Laser, 321, 333 
Spectrograph, 325, 328, 339 
Spectroscopist, 326 
Spectroscopy Laboratory, 326, 332 

(also see Massachusetts Institute of Tech¬ 
nology) 


Speculum coating for grating blanks, 326 
Sperry Gyroscope Co., 325 
Spindle, 226-227, 252, 280 
alignment with, 294 
housing design, 264-265, 270 
housing travel, 267-268 
orientation, 291 
overhead, type, 260-262, 269 
plain-bearing, 226, 265 
roller, 227, 259 

rotating-table type, 223, 260, 270 
rotation of, 267 
trueness of rotation, 224-227 
Universal Measuring Machine, 263 ff. 

Spin Table, 112-113, 235, 259-260 
Spindle housing Universal Measuring Machine, 
182, 286 

Spindle, measuring types, 259 ff. 
overhead, 260-262 
rotating table, 260 
Springfield Armory, 146, 150 
Square, T-square, 87 

Squareness, 34-35, 79-80, 86-87, 260-262, 
268-270 

Squareness of travel, 48-49 
Squash, check of cross-slide, 78-79 
Stability, dimensional, 16-19, 30, 101, 106, 121, 
155, 167, 187-189, 236, 320 
heat treatment, 235-237 
Standard, International, 107-108 
Bird, 104 

Standardization, 151, 155 
Standards: 

angle, 204 ff., 229 ff. 
gaging, 129 ff. 
krypton, 117 
length, 11 
physical, 114 ff. 

Standards bureaus, 94, 205, 236 
Standards, early: 

America, 106-107 
Egypt, 101-102 
England, 104 
France, 103 
Germany, 103-104 
Greece, 102-103 
Rome, 102-103 
Standards Room, 175 ff. 

(also see Moore Special Tool Co.) 

State University of New York, Stony Brook, 
186, 321, 335, 337 
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Steam engine, 141 ff. 

Newcomen, 137 
Steam hammer, 142 
Steel, early manufacturing, 138, 150 
Step Gage, 165-167, 175 ff., 182, 195, 324 
design, 184-185 

inspected by the bureaus, 183-184 
Moore masters, 175-176, 179-181, 183, 184 
185 

subdividing, 183 

Stepping motor, 220-221, 322, 324, 336 
Stewart-Warner Co., 318 
Stiction, 319, 336 

Stocker & Yale visual comparator, 298 
Straightness, geometry, 34, 40 ff., 47-50 esp., 
72-74, 76, 77, 80-86, 264, 267-270, 285, 
296, 334 

round shapes, barrel shape, 269 
bellmouth, 269 
hour glass, 269 
of straightedge, 317, 323 
Straightedge, 27-28, 31 ff., 82 ff. 
box-type, 33, 323, 334 
master, 316-317, 334 
nitralloy, 83, 317 
thickness, 317 
Strang, A. G„ 97, 111 

Stratification, 19, 50, 118, 171-173, 179, 320 
Stratton, Samuel W., 158 

Strip recorder, Universal Measuring Machine, 
316-317 

Stroke, George W., 186, 321, 325, 331-333, 
335-338 

Strong, John, 329, 330-331 
Surface finish, 125-128, 130, 136 
deformation, 131 ff. 
relation to roundness, 272-274 
(see also surface finish) 

Surface gage: (see indicator and stand) 
straddle, 28 

Surface plates, 14 ff., 129, 214, 271, 284, 311, 
321, 323 

box-type, 14, 21-22, 322 
cast iron, 24, 29 
granite, 23, 28-29, 119, 323 
master, 321, 323 

moisture, effect on flatness of, 29-30 

rectangular, 21 

ribbing, 21-22, 26 

rigidity of, 30 

round, 24 

support of, 22-23, 26, 35, 165 
square, 21-22, 24 ff. 
welded steel, 21 

Inspected on Universal Measuring Machine, 
323 


Surface texture, 28, 126 
“ASA B 46.1'’, 126 
flaws, 127 
lay, 127, 129-130 
micro-inch, 127 
roughness, 127, 129 
roughness, height, 127 
roughness, width, 127 
roughness, width cutoff, 127 
stylus, 128-129 
surface symbol, 127 
tracer-type analysis, 128-129 
waviness, 127, 129 
waviness height, 127 
waviness width, 127 
(see also surface texture) 

Sweden, 154 ff. 

Systeme Palmer, 151 


Table, Universal Measuring Machine, 62, 80 ff., 
86 ff. 

Tailstock unit, 227, 228, 312, 313 
Talysurf, 126 

Taper, measurements of, 204, 261-262, 266, 268, 
269, 270, 314 

Target mirror in autocollimation, 73-74, 82 
(see also mirror, use with autocollimator) 
Taylor-Hobson, Division of Rank Precision 
Industries, 126 
Telescope, 113, 117, 338 

(see also Diffraction gratings) 

Temperature, 50, 115 ff., 263 
control of, 168 ff. 
distortion, 283-284 

environmental effect, 18-19, 30-33, 118-120 
68° F., importance of, 117-120, 176 ff. 
Templates, 228, 319 
Terrien, J., 95 
Ternstrom, Ernst, 156 
Theodolites, 204 
Thermocouples, 180 

Thompson, Stephen W., 318, 331-333, 338 

Thorium, 327 

Timex watch, 285 

Toise, du Grand Chatelet, 103 

Tolerance, true position, 298-299 

Tomlinson, G. A., 228, 230 


Traceability, 152, 153, 156, 166-167, 205 
Tracing, polar, 264, 270-271, 273-275, 283, 314, 
316 

Tracing, surface condition, 124-127 
Tracing, straightness and thickness, 316-317 
Tracing, lead screw lead, 190-191 
Transducer of electronic indicator, 325 
Transferal of accuracy, 180 ff. 
end standard to step gage, 179, 180 
method of, 182-183 
Travel, machine: straightness, 47-48 
squareness of, 48-49 
Tresca design, 108, 114 
Triangle, 210 ff. 
right, 210-211 
shape, 210 
size, 210 

Troughton Scale, 107 
True position tolerancing, 298-299 
Trueness of rotation, 259, 263-264, 313 
effect on angular measurement, 224-225 
Tsugami Co., 135 

Turbine, inspection on Universal Measuring 
Machine, 313 
Turin Museum, 101 
Turley, George, 285 
Twist, 26-28, 44-46, 59, 76, 80-81 
(see also Parallelism, vertical) 

Twist gage, 26-28, 76, 78 
Tyndall, Professor, 155 


United States Coast and Geodetic Survey, 107, 
108 

United States Geological Survey, 318 
United States Standard of Roundness, 276 
U. S. Time Corp., 285 
Universal Measuring Machine, 

angular measuring instruments, use of, 310 ff. 

applications, 310 ff. 

controls, 286 

designs, special, 324 ff. 

manufacturing, benefits from, 282 ff. 

Model No. iy 2 , 298, 308 ff. 

Model No. 3, 280, 298-299, 308, 319, 333 
Model No. 4, 318 ff., 333-334, 338 
Model No. 5, 281, 318 ff. 
numerical control of, 319, 325 
readout, 309-310 
spindle, 252 
table, 270-272 
techniques, 287-310 
Z axis models, 319, 321, 322, 323 
Uranium, 327 







V-block, 69, 257-258, 272 
adjustable, 257 
fixed angle, 257 
master, 316 

Vandivert, William, 100 
Van Keuren Co., 112, 255-256 
Vaucanson, Jacques de, 139, 142 
Verbruggen, Jan, 139 

Vibration, effect on accuracy, 115, 120, 326, 
333-334 

Victory, Frederick C., 11, 124, 162, 166-167, 
223, 287, 295, 299 
Vitruvius, 9, 11, 102-103 
von Fraunhofer, Joseph, 327 
Vu-point readout, 309 


Wade, Herbert T., 106 
Wadsworth, Decius, 146 
Washington, George, 106 
Watt, James, 140, 141 
Way design, 44-45, 54-57, 142, 324 
Wear, surface plate, 30 

Wear, effect on geometric accuracy, 50, 51, 53, 
54, 55, 56, 57 

Wear, of scraping masters, 59 
Wear, of master laps, 67-68 
Wear, of lead screw, 162, 185, 187-188 
Wear, of lead screw nut, 191 
Wear, of length standard, 104, 117 
Wear of 1440 Index, 235 

Wear, of mechanical compensators 162,185, 218 
Webster, Ambrose, 150, 151 
Weights & measures: 

American, 106 
Anglo-Saxon, 104 
Babylonian, 99, 101-102 
Egyptian, 98-101 
Indus, 98-99 
Metric, 104 
English, 106 

Weights and Measures Act (1963), 106 
Wellington, Duke of, 149 
Whitney, Amos, 152 
Whitney, Eli, 107, 146, 280 
milling machine, 147, 149 


Whitworth, Joseph, 142-144, 151, 329 
Wiley, Robert S., 328, 333 
Wilkinson, John, 140 141 

cannon boring machine, 140-141, 149 
William I, 104 
Wood, Remsen W., 326, 327 
Woodworth, Circular Tables, 295 
Woodworth, Joseph, 159-160 
Woolwich Arsenal, 139, 142 
Workpiece, support, 282 
Worm & gear, 217, 218, 220 
manufacturing of, 223-225 
Worthen, W., 257 

Wringing, 115-116, 136, 155, 302-303 
Wringing film thickness, 136, 308 
Wringing with gage blocks, 115-116 


Yard, 103 

Baily’s line, 105-106 
Bronze No. 11, 107 
Imperial, 106 108, 114, 121, 153 
King Henry, 104, 114 
Queen Elizabeth (of 1760), 104, 114 
Standard of IT. S„ 152 
Troughton Scale, 107 
Troughton & Simms, 104, 107 
Yaw table, 334 

Yoke, measuring machine, 264-265 
Young, T. R., 108, 111, 136 
Young’s Modulus, 123, 130 


Zonal method, defining roundness, 273-276 






